stix2generator.generation package

Submodules

stix2generator.generation.constraints module

class stix2generator.generation.constraints.PresenceCoconstraint(property_names, constraint_type)

Bases: object

ALL = 1

AT_LEAST_ONE = 2

CONSTRAINT_TYPE_STRING_MAP = {0: 'one', 1: 'all', 2: 'at-least-one'}

ONE = 0

STRING_CONSTRAINT_TYPE_MAP = {'all': 1, 'at-least-one': 2, 'one': 0}

can_satisfy_without_refs()

Determine whether it is possible to satisfy this presence co-constraint with only non-reference properties.

choose_properties(probability, minimize_ref_properties)

Choose some properties from the group, according to its constraint type.

Parameters

• probability – This is used only for the at-least-one constraint type. For that type, one is randomly chosen as the required property; the rest are optional and included with this probability.

• minimize_ref_properties – Modify how property selection is done: if True, minimize reference properties in the selection. Reference properties may still be chosen, if there is no other way to satisfy the constraint.

Returns

A list of property names.

static to_constraint_type(constraint_type)

class stix2generator.generation.constraints.ValueCoconstraint(prop_name_left, operator, prop_name_right)

Bases: stix2generator.generation.constraints.ValueConstraintOperators

Represents a co-constraint between two properties. This just stores two property names and an operator. When the value of one property becomes known, a constraint can be derived for the other property.

get_constraint(prop_name, prop_value)

Given the value of one property of this co-constraint, derive a constraint on the other.

Parameters

• prop_name – The name of one property of this co-constraint

• prop_value – The value of prop_name

Returns

A ValueConstraint object representing a constraint on the other property.

Raises

ValueError – If the property name given isn’t a part of this co-constraint.

get_other_property(prop_name)

involves_property(prop_name)

class stix2generator.generation.constraints.ValueConstraint(operator, value)

Bases: stix2generator.generation.constraints.ValueConstraintOperators

Represents a constraint on a property. E.g. that it be less than some value. Currently this just consists of an operator and a value. The relationship to the property to be constrained is maintained externally, so it is not stored in these instances.

class stix2generator.generation.constraints.ValueConstraintOperators

Bases: object

A mixin/base-class for the constraint classes, which contains some convenience constants et al for operators.

EQ = 0

GE = 3

GT = 2

LE = 5

LT = 4

NE = 1

OPERATORS = range(0, 6)

OPERATOR_STRING_MAP = {0: '=', 1: '!=', 2: '>', 3: '>=', 4: '<', 5: '<='}

REVERSE_OPERATORS = (0, 1, 4, 5, 2, 3)

STRING_OPERATOR_MAP = {'!=': 1, '<': 4, '<=': 5, '=': 0, '>': 2, '>=': 3}

static to_operator(operator)

Normalize a value to an operator constant. If a string is passed, convert it to an operator constant. Otherwise, assume it is already an operator constant and just return it unchanged.

Parameters

operator – An operator string or operator constant

Returns

An operator constant

stix2generator.generation.constraints.make_value_coconstraint(value_coconstraint_expr)

Make a ValueCoconstraint object from a co-constraint expression. The expression syntax is simple: <prop_name> <op> <prop_name>. E.g. “a < b”.

Parameters

value_coconstraint_expr – The expression, as a string

Returns

The ValueCoconstraint object

Raises

ValueError – if the expression is invalid

stix2generator.generation.generate_stix module

stix2generator.generation.generate_stix.main()

stix2generator.generation.generate_stix.nonnegative_int(arg)

Verify/convert arg to a non-negative integer.

Parameters

arg – A commandline arg as a string

Returns

The integer

Raises

argparse.ArgumentTypeError – If the arg can’t be converted or is out of range

stix2generator.generation.generate_stix.parse_args()

stix2generator.generation.generate_stix.positive_int(arg)

Verify/convert arg to a positive integer.

Parameters

arg – A commandline arg as a string

Returns

The integer

Raises

argparse.ArgumentTypeError – If the arg can’t be converted or is out of range

stix2generator.generation.generate_stix.probability(arg)

Verify/convert arg to a probability value in the range [0, 1].

Parameters

arg – A commandline arg as a string

Returns

The probability as a float

Raises

argparse.ArgumentTypeError – If the arg can’t be converted or is out of range

stix2generator.generation.object_generator module

class stix2generator.generation.object_generator.Config(**kwargs)

Bases: stix2generator.generation.Config

Simple config class for the ObjectGenerator.

Most properties are self-explanatory. Some notes:

• The number settings apply to both “number” and “integer” specs

• optional_property_probability is the probability of inclusion of optional properties. Must be a number in [0, 1].

• Setting minimize_ref_properties to True will cause the generator to attempt to minimize reference properties in generated objects, while still satisfying specification constraints. If a reference property is necessary to satisfy a constraint, it will be included regardless of this setting.

  This setting overrides optional_property_probability for reference properties: if minimizing reference properties and optional_property_probability is 1 (meaning include all optional properties), optional reference properties may nevertheless be omitted.

class stix2generator.generation.object_generator.ObjectGenerator(spec_registry=None, semantic_providers=None, config=None)

Bases: object

Instances of this class generate JSON data from a specification. More specifically, JSON-serializable Python structures are generated, which can be converted to JSON text. So JSON objects are generated as dicts; arrays as lists, etc.

property config

Get this generator’s config object.

generate(spec_name, expected_type=None, spec_name_stack=None, value_constraint=None)

Generate a value based on a specification identified by name. The name is looked up in this generator’s registry.

Parameters

• spec_name – The specification name

• expected_type – If the named spec should produce a particular JSON type, that type. If it doesn’t matter, pass None. This can be used to identify reference errors, where the reference is to a specification for the wrong type of thing.

• spec_name_stack – A stack of previously-visited specification names, used for reference loop detection. Pass None to start a new stack.

• value_constraint – A ValueConstraint instance representing some additional constraint to be honored by the generator. This is derived from a value co-constraint expression. If None, there is no additional constraint.

Returns

The generated value

Raises

• stix2generator.exceptions.CyclicSpecificationReferenceError – If a circular reference chain is detected

• stix2generator.exceptions.SpecificationNotFoundError – If the named specification is not found in the registry

• stix2generator.exceptions.ObjectGenerationError – For many other types of generation errors

generate_array(array_spec, spec_name_stack=None)

Generate a JSON array from the given specification.

Parameters

• array_spec – A JSON array specification

• spec_name_stack – Specification name stack, for reference loop detection. If None, use an empty stack.

Returns

A list

Raises

stix2generator.exceptions.ObjectGenerationError – If a generation error occurs

generate_boolean(boolean_spec, spec_name_stack=None)

Generate a boolean from the given specification.

Parameters

• boolean_spec – A boolean specification (ignored; there’s nothing to configure for now)

• spec_name_stack – A specification name stack, for reference loop detection. Unused but included for API compatibility with object/array generators.

Returns

True or False

generate_from_spec(spec, expected_type=None, spec_name_stack=None, value_constraint=None)

Generate a value based on the given specification, which need not exist under any particular name in this generator’s registry.

Parameters

• spec – The specification, as parsed JSON

• expected_type – If the spec should be for a particular JSON type, that type. If it doesn’t matter, pass None.

• spec_name_stack – A stack of previously-visited specification names, used for reference loop detection. Pass None to start a new stack.

• value_constraint – A ValueConstraint instance representing some additional constraint to be honored by the generator. This is derived from a value co-constraint expression. If None, there is no additional constraint.

Returns

The generated value

Raises

• stix2generator.exceptions.UnrecognizedJSONTypeError – If given a non-const dict spec whose declared type is not recognized as a JSON type, or if expected_type is given and not a recognized JSON type.

• stix2generator.exceptions.TypeMismatchError – If expected_type is given and the spec type doesn’t match.

• stix2generator.exceptions.ObjectGenerationError – For various ways the given spec is invalid. Other types of errors are also wrapped/chained from this exception type (if possible) so that we get decoration with extra info from higher stack frames, which is useful for diagnosing where those problems occur.

generate_integer(integer_spec, spec_name_stack=None)

Generate an integer from the given specification.

Parameters

• integer_spec – An integer specification

• spec_name_stack – A specification name stack, for reference loop detection. Unused but included for API compatibility with object/array generators.

Returns

An int

Raises

stix2generator.exceptions.ObjectGenerationError – If a generation error occurs

generate_null(null_spec, spec_name_stack=None)

Generate null (None).

Parameters

• null_spec – A null specification (ignored; there’s nothing to configure for now)

• spec_name_stack – A specification name stack, for reference loop detection. Unused but included for API compatibility with object/array generators.

Returns

None

generate_number(number_spec, spec_name_stack=None)

Generate a number (float) from the given specification.

Parameters

• number_spec – A number specification

• spec_name_stack – A specification name stack, for reference loop detection. Unused but included for API compatibility with object/array generators.

Returns

A float

Raises

stix2generator.exceptions.ObjectGenerationError – If a generation error occurs

generate_object(object_spec, spec_name_stack=None)

Generate a JSON object from the given specification.

Parameters

• object_spec – A JSON object specification

• spec_name_stack – Specification name stack, for reference loop detection. If None, use an empty stack.

Returns

A dict

Raises

stix2generator.exceptions.ObjectGenerationError – If a generation error occurs

generate_string(string_spec, spec_name_stack=None)

Generate a string from the given specification.

Parameters

• string_spec – A string specification

• spec_name_stack – A specification name stack, for reference loop detection. Unused but included for API compatibility with object/array generators.

Returns

A string

Raises

stix2generator.exceptions.ObjectGenerationError – If a generation error occurs

property spec_names

Get a read-only iterable view of this generator’s supported spec names.

Returns

An iterable of spec names

stix2generator.generation.pattern_generator module

class stix2generator.generation.pattern_generator.Config(**kwargs)

Bases: stix2generator.generation.Config

STIX pattern generator config settings.

min/max_pattern_size: Bounds on pattern size, in terms of the total number

of simple comparison expressions across all observation expressions.

min/max_repeat_count: bounds on the repeat count for the REPEATS qualifier.

min/max_within_count: bounds on the number of seconds for the WITHIN

qualifier.

probability_qualifier: Probability a given observation expression (at any

nesting level) will get a random qualifier.

probability_continue_path_through_ref: When randomly generating an object

path, determines how likely the path will continue through a reference property to another object.

probability_index_star_step: When randomly generating an object path,

determines how likely list index steps will use ‘*’.

class stix2generator.generation.pattern_generator.PatternGenerator(object_generator, stix_version, config=None)

Bases: object

Instances of this class generate random STIX patterns.

generate()

Generate a random STIX pattern.

Returns

A pattern string

generate_ast()

Generate a random STIX pattern as an AST.

Returns

A pattern AST

stix2generator.generation.reference_graph_generator module

class stix2generator.generation.reference_graph_generator.Config(**kwargs)

Bases: stix2generator.generation.Config

Config settings for reference graph generation.

• max_depth: about how many steps maximum from the “seed” object are we

  allowed to get. This affects the maximum size of the graph. It can’t be strictly enforced, since one can’t generally guarantee that a generated object will have no reference properties.

• graph_type: what style of graph to create. See the GraphType enum.

• probability_reuse: when creating an object graph, in those situations

  where we can choose to reuse an already-created object, this determines how likely we are to choose to reuse.

• inverse_property_constraints: how to handle inverse property constraints.

  See the InversePropertyConstraintEnforcement enum.

class stix2generator.generation.reference_graph_generator.GraphType(value)

Bases: enum.Enum

Styles of graph:

• TREE: No object “reuse”, i.e. in-degree for all nodes is never greater

  than one. Exceptions will occur for objects with fixed IDs, e.g. TLP marking definitions.

• DAG: Directed acyclic graph: objects may be reused as long as no cycle is

  created.

• RANDOM: Objects are randomly reused without regard to cycles.

The acyclic graph topologies are still subject to inverse property cycles. So depending on how inverse property constraints are enforced, it may not be possible to have a truly acyclic graph.

DAG = 1

RANDOM = 2

TREE = 0

class stix2generator.generation.reference_graph_generator.InversePropertyConstraint(object_type1, prop_name1, object_type2, prop_name2)

Bases: object

Instances represent an “inverse” property constraint. This type of constraint means that reference properties in each of two objects are “inverses” with respect to each other: if you set one to refer to a particular object, the inverse property of the other object must be set to refer to the first object. They must refer to each others’ objects. An example of this is directory/content_refs and file/parent_directory_ref. This necessarily implies cycles in the graph.

Constraint specification is simple: you just give two STIX object types and the property name from each which are inverses with respect to each other. This specification is also symmetric: (type1, A, type2, B) means the same as (type2, B, type1, A), where A and B are property names.

Application of the constraint to a pair of objects however, may not be so simple. The type and property names need not be distinct. This can result in ambiguity in applying the constraint. When type names are equal, an instance of the type may have both constraint properties. Then, the constraint can apply in two different ways. For example, consider the process SCO, which has child_refs and parent_ref properties, which are inverses:

process1:

child_refs=[2] parent_ref=2

process2:

child_refs=[3] parent_ref=1

process1’s child_refs refers to process2, so the constraint says process2’s parent_ref must refer to process1. It does, so the constraint holds in that case. But process1’s parent_ref refers to process2, and process2’s child_refs does not refer to process1. So the constraint does not hold in that case. Both objects have both constraint properties, so the constraint effectively applies twice, and in each case independently, it may or may not hold. So it’s not sufficient to ask whether an arbitrary inverse property constraint holds between two arbitrary objects. One must somehow express how to apply the constraint, in cases where there is ambiguity.

So method signatures for this class include the ref_prop parameter for this purpose. It expresses which constraint property in the first object (src_obj) you are interested in. In results in asymmetry in the application of the constraint, despite the symmetry of the constraint itself. For example, the holds() method is essentially checking for an inverse property cycle between two objects. With respect to the (type1, A, type2, B) example above, invoking as holds(src_obj, A, dest_obj) means checking for the cycle:

src_obj -> A -> dest_obj -> B -> src_obj

whereas invoking as holds(src_obj, B, dest_obj) means checking for the cycle:

src_obj -> B -> dest_obj -> A -> src_obj

and those cycles are different (assuming A and B are different properties).

enforce(src_obj, ref_prop, dest_obj)

Assuming applicability of this constraint (see is_applicable()), enforce this constraint on dest_obj. This will force a property of dest_obj to refer to src_obj. For a list-valued constraint property in dest_obj, an arbitrary element will be forced to src_obj’s ID.

Parameters

• src_obj – An object

• ref_prop – A property name

• dest_obj – Another object

holds(src_obj, ref_prop, dest_obj)

Assuming applicability of this constraint (see is_applicable()), checks whether dest_obj refers back to src_obj via the inverse property (of ref_prop).

Parameters

• src_obj – An object

• ref_prop – A property name

• dest_obj – Another object

Returns

True if the constraint holds; False if it does not

is_applicable(src_obj, ref_prop, dest_obj)

Given a source/dest object and a ref property name, determine whether this constraint would constrain an inverse property of dest_obj. To be applicable, the source object type and ref_prop must match with one of the type/property pairs of this constraint, dest_obj’s type must match with the other type, it must possess the inverse property, and the source object must actually refer to dest_obj via ref_prop. dest_obj need not refer back to src_obj via the inverse property (if this constraint were actually enforced, it would be).

Parameters

• src_obj – An object

• ref_prop – A property name

• dest_obj – Another object

Returns

True if this constraint matches the props/types of the objects and would constrain a property of dest_obj; False if not.

is_applicable_types(src_type, ref_prop, dest_type)

Applicability check based only on types, not actual objects. If an instance of src_type were linked to an instance of dest_type via the source object’s ref_prop property, determine whether this constraint would constrain the dest instance (assuming it had the inverse property).

Parameters

• src_type – An object type

• ref_prop – A property name

• dest_type – Another object type

Returns

True if an instance of dest_type would be constrained; False if not.

property object_type1

property object_type2

property prop_name1

property prop_name2

class stix2generator.generation.reference_graph_generator.InversePropertyConstraintEnforcement(value)

Bases: enum.Enum

An inverse property constraint is a constraint implied by two properties with an “inverse” relationship to each other. (See the InversePropertyConstraint class.) This enum determines how we should handle them.

ENFORCE: Enforce them. Where inverse properties occur, ensure they

refer to each other’s objects. This means that acyclic graph topologies are not truly possible, because inverse properties will always result in a cycle.

DELETE: Delete one of the properties. This ensures inverse property

cycles will not occur. This is done naively, without regard to whether the property being deleted may have been required by spec. So it could result in invalid objects, depending on how they were designed.

IGNORE: Pretend there are no constraints. This treats inverse properties

as any other property. This can result in nonsensical/contradictory semantics in the content. E.g. directory1 contains file1, whose parent directory is directory2. This means directory1 is declared to contain file1, but file1 claims to be in directory2.

DELETE = 1

ENFORCE = 0

IGNORE = 2

class stix2generator.generation.reference_graph_generator.ReferenceGraphGenerator(object_generator, config=None, stix_version='2.1')

Bases: object

Instances of this class can generate object graphs such that the nodes are objects and edges correspond to reference properties of those objects. These graphs do not use SROs. Generation is designed to prevent any dangling references in the resulting graph.

property config

generate(seed_type=None, preexisting_objects=None)

Generate a reference graph seeded with an object of the given type. For graph types which allow reuse, preexisting_objects provides a way to reuse pre-existing objects. This can be helpful when larger amounts of graph content is built up incrementally, so that new regions of content can connect with existing regions, instead of each invocation producing its own disconnected “island” of content.

Parameters

• seed_type – A STIX type, STIXTypeClass enum, or None. Generation is done by “building out” from a seed object. This parameter determines the type of the seed object. If None, a STIX SDO or SCO is chosen at random.

• preexisting_objects – The pre-existing STIX content, as either a list of objects or mapping from ID to object.

Returns

A 2-tuple including (1) The ID of the object which was created as the seed, and (2) the generated content, as a mapping from ID to object. If preexisting_objects was given as a map, the same map is returned, updated with the new content. Otherwise, a new map is created and returned. It will contain all of the data from preexisting_objects plus the new content.

stix2generator.generation.relationships module

class stix2generator.generation.relationships.Relationship(src_type, rel_type, target_type)

Bases: tuple

property rel_type

Alias for field number 1

property src_type

Alias for field number 0

property target_type

Alias for field number 2

stix2generator.generation.semantics module

class stix2generator.generation.semantics.FakerSemantics(faker, faker_names=None)

Bases: stix2generator.generation.semantics.SemanticsProvider

A semantics provider which uses a faker. The semantics names are taken directly from the faker object’s providers’ method names. You can pass arguments into the faker method by adding additional properties to the specification. They will be passed as keyword args.

create_semantic(spec, generator, constraint=None)

Create a value according to the given semantic spec. For convenience, this default implementation tries to invoke a method named the same as the semantic name, except if the name contains hyphens. If it contains hyphens, they are translated to underscores first, so it is easier to define the methods. (Can’t easily have a method name with a hyphen.) Subclasses can override this to do something else.

Parameters

• spec – The semantic spec, which must be a mapping with a “semantics” property giving the name of the semantic. Of course, that name should be one of the names returned by get_semantics()! Other properties may be used by the implementation, e.g. to influence how generation is done.

• generator – The generator through which this provider was invoked. This allows implementations to delegate to other specifications as needed. (But see the note in the class docstring.)

• constraint – A ValueConstraint instance representing some additional constraint to be honored by the generator. This is derived from a co-constraint expression. If None, there is no additional constraint.

Returns

The generated value

get_semantics()

Get a list of the semantics provided by the provider. Subclasses must override this.

Returns

The list of semantics (list of strings which are the names of the semantics).

class stix2generator.generation.semantics.STIXSemantics

Bases: stix2generator.generation.semantics.SemanticsProvider

Some STIX-specific custom semantics.

SEMANTICS = ['stix-id', 'stix-timestamp', 'stix-pattern', 'sha256', 'sha512', 'sha3_256', 'sha3_512', 'observable-container']

get_semantics()

Get a list of the semantics provided by the provider. Subclasses must override this.

Returns

The list of semantics (list of strings which are the names of the semantics).

observable_container(spec, generator, constraint)

Generate an “observable container” object. This is used as the value of the “objects” property of STIX 2.0-style observed-data objects. That property is deprecated in STIX 2.1.

sha256(spec, generator, constraint)

Generate a random SHA256 hash.

sha3_256(spec, generator, constraint)

Generate a random SHA3-256 hash.

sha3_512(spec, generator, constraint)

Generate a random SHA3-512 hash.

sha512(spec, generator, constraint)

Generate a random SHA512 hash.

stix_id(spec, generator, constraint)

Create a STIX ID. The spec requires the “stix-type” property, which determines the “<type>–” prefix.

stix_pattern(spec, generator, constraint)

Create a random STIX pattern.

stix_timestamp(spec, generator, constraint)

Create a random STIX timestamp which honors the given constraint.

class stix2generator.generation.semantics.SemanticsProvider

Bases: object

Abstract base class for semantics providers. Instances generate data given an abstract semantic name and some additional settings.

NOTE: An implementation is provided the generator it was invoked through so that it may leverage auto-generation to create parts of its values. Avoid invoking a named specification from the given generator (e.g. via the “generate” method). To be reusable across generators, a provider can’t make assumptions as to which named specifications exist. Also, you don’t know what context the semantic is being invoked within: if the named specification is already in the midst of generation, you will cause infinite recursion.

Semantics implementations should only generate from specifications which don’t contain any references.

create_semantic(spec, generator, constraint=None)

Create a value according to the given semantic spec. For convenience, this default implementation tries to invoke a method named the same as the semantic name, except if the name contains hyphens. If it contains hyphens, they are translated to underscores first, so it is easier to define the methods. (Can’t easily have a method name with a hyphen.) Subclasses can override this to do something else.

Parameters

• spec – The semantic spec, which must be a mapping with a “semantics” property giving the name of the semantic. Of course, that name should be one of the names returned by get_semantics()! Other properties may be used by the implementation, e.g. to influence how generation is done.

• generator – The generator through which this provider was invoked. This allows implementations to delegate to other specifications as needed. (But see the note in the class docstring.)

• constraint – A ValueConstraint instance representing some additional constraint to be honored by the generator. This is derived from a co-constraint expression. If None, there is no additional constraint.

Returns

The generated value

get_semantics()

Get a list of the semantics provided by the provider. Subclasses must override this.

Returns

The list of semantics (list of strings which are the names of the semantics).

property logger

Convenience property to get a logger named after the instance’s module and class name.

Returns

A logger instance

stix2generator.generation.stix_generator module

class stix2generator.generation.stix_generator.Config(**kwargs)

Bases: stix2generator.generation.Config

STIX generator config settings.

min/max_relationships: Limits on the number of SROs created. This

influences (but does not solely determine) the size of the resulting graph.

probability_reuse: One endpoint of every SRO connects to existing content,

to avoid partitioning the graph. This setting determines how likely the other endpoint(s) are to refer to existing content.

probability_sighting: When each SRO is added, with this probability, a

sighting will be attempted instead of a “plain” relationship. A plain relationship is still used as a fallback if a sighting cannot be created.

complete_ref_properties: When an object is created, whether or not to

leave its reference properties dangling. Applies only to non-SRO-endpoint reference properties.

class stix2generator.generation.stix_generator.STIXGenerator(object_generator, ref_graph_generator=None, config=None, stix_version='2.1')

Bases: object

Instances generate STIX content: STIX objects related by SROs and embedded relationships.

property config

generate(seed_type=None, preexisting_objects=None)

Generate a STIX graph seeded with an object of the given type. When object reuse is allowed, preexisting_objects provides a way to reuse pre-existing objects. This can be helpful when larger amounts of graph content is built up incrementally, so that new regions of content can connect with existing regions, instead of each invocation producing its own disconnected “island” of content.

Parameters

• seed_type – A STIX type, STIXTypeClass enum, or None. Generation is done by “building out” from a seed object. This parameter determines the type of the seed object. If None, a STIX SDO is chosen at random.

• preexisting_objects – The pre-existing STIX content, as either a list of objects or mapping from ID to object.

Returns

the generated content, as a mapping from ID to object. If preexisting_objects was given as a map, the same map is returned, updated with the new content. Otherwise, a new map is created and returned. It will contain all of the data from preexisting_objects plus the new content.

Module contents

class stix2generator.generation.Config(**kwargs)

Bases: object

Simple config class which just holds a set of attributes and values.