cloudmarker.clouds package

A package for cloud plugins packaged with this project.

This package contains cloud plugins that are packaged as part of this project. The cloud plugins implement a function named read() that connects to remote data sources, typically cloud APIs, and yield data records.

Submodules

cloudmarker.clouds.azcloud module

Microsoft Azure cloud plugin to read Azure infrastructure data.

This module defines the AzCloud class that retrieves data from Microsoft Azure.

class cloudmarker.clouds.azcloud.AzCloud(tenant, client, secret, _max_subs=0, _max_recs=0)

Bases: object

Azure cloud plugin.

Create an instance of AzCloud plugin.

Note: The _max_subs and _max_recs arguments should be used only in the development-test-debug phase. They should not be used in production environment. This is why we use the convention of beginning their names with underscore.
Parameters:
  • tenant (str) – Azure subscription tenant ID.
  • client (str) – Azure service principal application ID.
  • secret (str) – Azure service principal password.
  • _max_subs (int) – Maximum number of subscriptions to fetch data for if the value is greater than 0.
  • _max_recs (int) – Maximum number of records of each type to fetch under each subscription.
done()

Perform clean up tasks.

Currently, this method does nothing because there are no clean up tasks associated with the AzCloud plugin. This may change in future.

read()

Return an Azure cloud infrastructure configuration record.

Yields:dict – An Azure cloud infrastructure configuration record.

cloudmarker.clouds.azvm module

Microsoft Azure virtual machine plugin to read Azure virtual machine data.

This module defines the AzVM class that retrieves virtula machine data from Microsoft Azure.

class cloudmarker.clouds.azvm.AzVM(tenant, client, secret, _max_subs=0, _max_recs=0)

Bases: object

Azure Virtual Machine plugin.

Create an instance of AzVM plugin.

Note: The _max_subs and _max_recs arguments should be used only in the development-test-debug phase. They should not be used in production environment. This is why we use the convention of beginning their names with underscore.
Parameters:
  • tenant (str) – Azure subscription tenant ID.
  • client (str) – Azure service principal application ID.
  • secret (str) – Azure service principal password.
  • _max_subs (int) – Maximum number of subscriptions to fetch data for if the value is greater than 0.
  • _max_recs (int) – Maximum number of virtual machines records to fetch for each subscription.
done()

Perform clean up tasks.

Currently, this method does nothing because there are no clean up tasks associated with the AzVM plugin. This may change in future.

read()

Return an Azure virtual machine record.

Yields:dict – An Azure virtual machine record.

cloudmarker.clouds.gcpcloud module

Google Cloud Platform (GCP) plugin to read GCP infrastructure data.

This module defines the GCPCloud class that retrieves data from Google Cloud Platform.

class cloudmarker.clouds.gcpcloud.GCPCloud(key_file_path, zone)

Bases: object

GCP cloud plugin.

Create an instance of GCPCloud plugin.

Parameters:
  • key_file_path (str) – Path of the service account key file for a project.
  • zone (str) – Zone of GCP Project, e.g., us-east1-b.
done()

Perform clean up tasks.

Currently, this method does nothing because there are no clean up tasks associated with the GCPCloud plugin. This may change in future.

read()

Return a GCP infrastructure configuration record.

Yields:dict – Firewall rule or VM instance configuration data.

cloudmarker.clouds.mockcloud module

Mock cloud plugin for testing purpose.

class cloudmarker.clouds.mockcloud.MockCloud(record_count=10, record_types=('foo', 'bar'))

Bases: object

Mock cloud plugin for testing purpose.

Create an instance of MockCloud plugin.

This plugin generates mock records. The records generated contains three fields under three top-level keys that we also call “bucket keys”: raw, data, and type, as shown in the example below:

Example

Here is an example that shows that the records generated by this plugin with the default initialization parameters:

>>> from cloudmarker.clouds import mockcloud
>>> cloud = mockcloud.MockCloud()
>>> for record in cloud.read():
...     print(record['raw']['data'],
...           record['ext']['record_type'],
...           record['com']['record_type'])
...
0 foo mock
1 bar mock
2 foo mock
3 bar mock
4 foo mock
5 bar mock
6 foo mock
7 bar mock
8 foo mock
9 bar mock

The three top-level keys, raw, ext, and com represent the names of the three buckets under which various data attributes are kept. While this is only a mock plugin, but in an actual cloud plugin implementation, the meaning of these buckets are as follows:

  • raw: The value for the raw key is a dict object that represents the actual data object obtained from a cloud in its original form. No modifications should be done to the object obtained from the cloud.
  • ext: The value for the ext key is a dict object which contains key-value pairs for any additional cloud-specific metadata that need to be stored. The data in this bucket is also known as extended metadata.
  • com: The value for the com key is a dict object which contains key-value pairs for any metadata that is common to all clouds.
Parameters:
  • record_count (int) – Number of mock records to generate.
  • record_types (tuple) – A tuple of strings that represent the different record types to be generated.
done()

Perform cleanup work.

Since this is a mock plugin, this method does nothing. However, a typical cloud plugin may or may not need to perform cleanup work in this method depending on its nature of work.

read()

Generate a record.

This method creates and yields mock records.

In actual cloud implementations, this method would typically connect to the cloud, retrieve JSON objects using the cloud API, and yield those objects as dict objects.

Yields:dict – Mock record.