cloudmarker.clouds package

A package for cloud plugins packaged with this project.

This package contains cloud plugins that are packaged as part of this project. The cloud plugins implement a function named read() that connects to remote data sources, typically cloud APIs, and yield data records.

Submodules

cloudmarker.clouds.azcloud module

cloudmarker.clouds.azvm module

cloudmarker.clouds.gcpcloud module

cloudmarker.clouds.mockcloud module

Mock cloud plugin for testing purpose.

class cloudmarker.clouds.mockcloud.MockCloud(record_count=10, record_types=('foo', 'bar'))

Bases: object

Mock cloud plugin for testing purpose.

Create an instance of MockCloud plugin.

This plugin generates mock records. The records generated contains three fields under three top-level keys that we also call “bucket keys”: raw, data, and type, as shown in the example below:

Example

Here is an example that shows that the records generated by this plugin with the default initialization parameters:

>>> from cloudmarker.clouds import mockcloud
>>> cloud = mockcloud.MockCloud()
>>> for record in cloud.read():
...     print(record['raw']['data'],
...           record['ext']['record_type'],
...           record['com']['record_type'])
...
0 foo mock
1 bar mock
2 foo mock
3 bar mock
4 foo mock
5 bar mock
6 foo mock
7 bar mock
8 foo mock
9 bar mock

The three top-level keys, raw, ext, and com represent the names of the three buckets under which various data attributes are kept. While this is only a mock plugin, but in an actual cloud plugin implementation, the meaning of these buckets are as follows:

  • raw: The value for the raw key is a dict object that represents the actual data object obtained from a cloud in its original form. No modifications should be done to the object obtained from the cloud.
  • ext: The value for the ext key is a dict object which contains key-value pairs for any additional cloud-specific metadata that need to be stored. The data in this bucket is also known as extended metadata.
  • com: The value for the com key is a dict object which contains key-value pairs for any metadata that is common to all clouds.
Parameters:
  • record_count (int) – Number of mock records to generate.
  • record_types (tuple) – A tuple of strings that represent the different record types to be generated.
done()

Perform cleanup work.

Since this is a mock plugin, this method does nothing. However, a typical cloud plugin may or may not need to perform cleanup work in this method depending on its nature of work.

read()

Generate a record.

This method creates and yields mock records.

In actual cloud implementations, this method would typically connect to the cloud, retrieve JSON objects using the cloud API, and yield those objects as dict objects.

Yields:dict – Mock record.