Extensions
smc-python provides additional extensions to extend the base library. Extensions are installed as separate packages and will have the dependency on the base smc-python library.
Available extensions:
smc-python-monitoring
smc-python-monitoring
smc-python-monitoring API provides a monitoring interface to the SMC to perform queries for dynamic engine components such as blacklists, connections, routes, vpn’s, users and logs.
Capabilities in the API implement the functionality found in the Log Server and engine level monitoring.
Query
A Query is the top level object used to construct parameters to make queries to the SMC.
Query is the parent class for all monitors in package smc_monitoring.monitors
Each monitor type will have it’s own predefined set of log fields that are considered ‘default’ for the query type. These will correlate closely to the default fields you will see in the Management Client component of the SMC when viewing the same information (Connections, VPN SAs, Blacklist, etc).
Each query also has a specific formatter which defines how the data is returned from
the query. Formatters are defined in smc_monitoring.models.formats
.
Each formatter type allows customization of the field_format and allows a value of ‘pretty’, ‘name’ or ‘id’. By default ‘pretty’ is used as the format which aligns with the column names in the Management Client monitoring views.
- class smc_monitoring.models.query.Query(definition=None, target=None, format=None, **sockopt)[source]
Query is the top level structure for controlling requests over the SMC websocket protocol. Any keyword arguments are passed through from inheriting classes are passed through as socket options for
smc_monitoring.wsocket.SMCSocketProtocol
.- Variables:
request (dict) – built request, eventually sent to socket
format (TextFormat) – format settings for query
- add_and_filter(*values)[source]
Add a filter using “AND” logic. This filter is useful when requiring multiple matches to evaluate to true. For example, searching for a specific IP address in the src field and another in the dst field.
See also
smc_monitoring.models.filters.AndFilter
for examples.- Parameters:
values – optional constructor args for
smc_monitoring.models.filters.AndFilter
. Typically this is a list of InFilter expressions.- Type:
list(QueryFilter)
- Return type:
- add_defined_filter(*value)[source]
Add a DefinedFilter expression to the query. This filter will be considered true if the
smc.monitoring.values.Value
instance has a value.See also
smc_monitoring.models.filters.DefinedFilter
for examples.- Parameters:
value (Value) – single value for the filter. Value is of type
smc_monitoring.models.values.Value
.- Type:
list(QueryFilter)
- Return type:
- add_in_filter(*values)[source]
Add a filter using “IN” logic. This is typically the primary filter that will be used to find a match and generally combines other filters to get more granular. An example of usage would be searching for an IP address (or addresses) in a specific log field. Or looking for an IP address in multiple log fields.
See also
smc_monitoring.models.filters.InFilter
for examples.- Parameters:
values – optional constructor args for
smc_monitoring.models.filters.InFilter
- Return type:
- add_not_filter(*value)[source]
Add a filter using “NOT” logic. Typically this filter is used in conjunction with and AND or OR filters, but can be used by itself as well. This might be more useful as a standalone filter when displaying logs in real time and filtering out unwanted entry types.
See also
smc_monitoring.models.filters.NotFilter
for examples.- Parameters:
values – optional constructor args for
smc_monitoring.models.filters.NotFilter
. Typically this is a list of InFilter expressions.- Type:
list(QueryFilter)
- Return type:
- add_or_filter(*values)[source]
Add a filter using “OR” logic. This filter is useful when matching on one or more criteria. For example, searching for IP 1.1.1.1 and service TCP/443, or IP 1.1.1.10 and TCP/80. Either pair would produce a positive match.
See also
smc_monitoring.models.filters.OrFilter
for examples.- Parameters:
values – optional constructor args for
smc_monitoring.models.filters.OrFilter
. Typically this is a list of InFilter expressions.- Type:
list(QueryFilter)
- Return type:
- add_translated_filter()[source]
Add a translated filter to the query. A translated filter syntax uses the SMC expression syntax to build the filter. The simplest way to see the syntax is to create a filter in the Logs view of the Management Client and right click->Show Expression.
Example to fetch a specific Situation from the Active Alerts:
t_filter.update_filter('$Situation==516') query = ActiveAlertQuery('Shared Domain', timezone='Berlin/Europe') query.update_filter(t_filter) for record in query.fetch_batch(): print(record) .. seealso:: :class:`smc_monitoring.models.filters.TranslatedFilter` for examples.
- Parameters:
values – optional constructor args for
smc_monitoring.models.filters.TranslatedFilter
- Type:
list(QueryFilter)
- Return type:
- execute()[source]
Execute the query with optional timeout. The response to the execute query is the raw payload received from the websocket and will contain multiple dict keys and values. It is more common to call query.fetch_XXX which will filter the return result based on the method. Each result set will have a max batch size of 200 records. This method will also continuously return results until terminated. To make a single bounded fetch, call
fetch_batch()
orfetch_raw()
.
- fetch_batch(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>, **kw)[source]
Fetch and return in the specified format. Output format is a formatter class in
smc_monitoring.models.formatters
. This fetch type will be a single shot fetch unless providing max_recv keyword with a value greater than the default of 1. Keyword arguments available are kw infetch_raw()
.- Parameters:
query_timeout (int) – length of time to wait on recieving web socket results (total query time).
inactivity_timeout (int) – length of time before exiting if no new entry.
max_recv (int) – for queries that are not ‘live’, set this to supply a max number of receive iterations.
formatter – Formatter type for data representation. Any type in
smc_monitoring.models.formatters
.
- Returns:
generator returning data in specified format
Note
You can provide your own formatter class, see
smc_monitoring.models.formatters
for more info.
- fetch_live(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>)[source]
Fetch a live stream query. This is the equivalent of selecting the “Play” option for monitoring fields within the Management Client. Data will be streamed back in real time.
- Parameters:
formatter – Formatter type for data representation. Any type in
smc_monitoring.models.formatters
.- Returns:
generator yielding results in specified format
- fetch_raw(**kw)[source]
Fetch the records for this query. This fetch type will return the results in raw dict format. It is possible to limit the number of receives on the socket that return results before exiting by providing max_recv.
This fetch should be used if you want to return only the result records returned from the query in raw dict format. Any other dict key/values from the raw query are ignored.
- Parameters:
max_recv (int) – max number of socket receive calls before returning from this query. If you want to wait longer for results before returning, increase max_iterations (default: 0)
query_timeout (int) – length of time to wait on recieving web socket results (total query time).
inactivity_timeout (int) – length of time before exiting if no new entry.
- Returns:
list of query results
- Return type:
- static resolve_field_ids(ids, **kw)[source]
Retrieve the log field details based on the LogField constant IDs. This provides a helper to view the fields representation when using different field_formats. Each query class has a default set of field IDs that can easily be looked up to examine their fields and different label options. For example:
Query.resolve_field_ids(ConnectionQuery.field_ids)
- update_filter(filt)[source]
Update the query with a new filter.
- Parameters:
filt (
smc_monitoring.models.filters.QueryFilter
) – change query to use new filter
- update_format(format)[source]
Update the format for this query.
- Parameters:
format (
smc_monitoring.models.formats
) – new format to use for this query
Models
The models package consists of the building blocks that make up a query.
Each module represents different class models that simplify adding things like filters, specifying values and formats.
Filters
Filters are used by queries to refine how results are returned.
QueryFilter is the top level ‘interface’ for all filter types. The filter
attribute of a QueryFilter provides access to the compiled query string used
to build the filter. Each QueryFilter also has an update_filter
method
that can be used to swap new filters in and out of an existing query.
Filters can be added to queries using the add_XXX methods of the query, or by building the filters and adding to the query using query.update_filter(). Filters can be swapped in and out of a query.
Examples:
Build a query to return all records of alert severity high or critical:
query = LogQuery(fetch_size=50)
query.add_in_filter(
FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH, Alerts.CRITICAL)])
If you prefer building your filters individually, it is not required to call the
add_XX_filter methods of the query. You can also insert filters by building the filter
and calling the update_filter
method on the query:
query = LogQuery(fetch_size=50)
query.update_filter(
InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53', 'TCP/80')])
You can also replace existing query filters with new filters to re-use the base level query parameters such as fetch_size, format style, time/date ranges, etc.
Replace the existing query filter with a different filter:
new_filter = InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53', 'TCP/80')])
query.update_filter(new_filter)
Note
it is also possible to update a filter by calling query.add_XX_filter methods multiple times. Each time will replace an existing filter if it exists.
For example, calling add_XX_filter methods multiple times to refine filter results:
query = LogQuery(fetch_size=50)
query.add_in_filter( # First filter query - look for alert severity high and critical
FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH, Alerts.CRITICAL)])
query.add_and_filter([ # Change filter to AND filter for further granularity
InFilter(FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH, Alerts.CRITICAL)]),
InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')])])
- class smc_monitoring.models.filters.AndFilter(*filters)[source]
Bases:
QueryFilter
An AND filter combines other filter types and requires that each filter matches. An AND filter is a collection of QueryFilter’s, typically IN or NOT filters that are AND’d together.
Example of fetching 50 records for sources matching ‘192.168.4.84’ and a service of ‘TCP/80’:
query = LogQuery(fetch_size=50) query.add_and_filter([ InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')]), InFilter(FieldValue(LogField.SERVICE), [ServiceValue('TCP/80')])])
- class smc_monitoring.models.filters.CILikeFilter[source]
Bases:
QueryFilter
A CILikeFilter is a case insensitive LIKE string match filter.
- class smc_monitoring.models.filters.CSLikeFilter[source]
Bases:
QueryFilter
A CSLikeFilter is a case sensitive LIKE string match filter.
- class smc_monitoring.models.filters.DefinedFilter(value=None)[source]
Bases:
QueryFilter
A Defined Filter applied to a query will only match if the value specified has a value in the audit record/s.
Show only records that have a defined Action (read as ‘match if action has a value’):
query = LogQuery(fetch_size=50) query.add_defined_filter(FieldValue(LogField.ACTION))
DefinedFilter’s can be used in AND, OR or NOT filter queries as well. Fetch the most recent 50 records for source 192.168.4.84 that have an application defined:
query = LogQuery(fetch_size=50) query.add_and_filter([ DefinedFilter(FieldValue(LogField.IPSAPPID)), InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')])])
- Parameters:
values (Value) – single value type to require on filter
- class smc_monitoring.models.filters.InFilter(left, right)[source]
Bases:
QueryFilter
InFilter’s are made up of two parts, a left and a right. An InFilter is considered a match if evaluation of the left part is equivalent to one of the elements of the right part. The left part of an InFilter is made up of a target of type
smc.monitoring.values.Value
. The right part is made up of a list of the same type.Search the Source field for IP addresses 192.168.4.84 or 10.0.0.252:
query = LogQuery(fetch_size=50) query.add_in_filter( FieldValue(LogField.SRC), [IPValue('192.168.4.84', '10.0.0.252')])
Reverse the logic and search for IP address 192.168.4.84 in source and dest log fields:
query = LogQuery(fetch_size=50) query.add_in_filter( IPValue('192.168.4.84'), [FieldValue(LogField.SRC, LogField.DST)])
InFilter’s are one of the most common filters and are often added to AND, OR or NOT filters for more specific matching.
- Parameters:
left (Values: any value type in
smc_monitoring.models.values
) – single value for leftmost portion of filterright (list(Values): any value type in
smc_monitoring.models.values
) – list of values for rightmost portion of filter
- class smc_monitoring.models.filters.NotFilter(*filters)[source]
Bases:
QueryFilter
A NOT filter provides the ability to suppress auditing based on a specific filter. A NOT filter is typically added to an AND filter to remove unwanted entries from the response.
Use only a NOT filter to a query and to ignore DNS traffic:
query = LogQuery(fetch_size=50) query.add_not_filter( [InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53')])])
The above example by itself is not overly useful, however you can use NOT filters with AND filters to achieve a logic like “Find source IP 192.168.4.68 and not service UDP/53 or TCP/80”:
query = LogQuery(fetch_size=50) not_dns = NotFilter( [InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53', 'TCP/80')])]) by_ip = InFilter( FieldValue(LogField.SRC), [IPValue('172.18.1.20')]) query.add_and_filter([not_dns, by_ip])
- class smc_monitoring.models.filters.OrFilter(*filters)[source]
Bases:
QueryFilter
An OR filter matches if any of the combined filters match. An OR filter is a collection of QueryFilter’s, typically IN or NOT filters that are OR’d together.
Example of fetching 50 records for sources matching ‘192.168.4.84’ or a service of ‘TCP/80’:
query = LogQuery(fetch_size=50) query.add_or_filter([ InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')]), InFilter(FieldValue(LogField.SERVICE), [ServiceValue('TCP/80')])])
- class smc_monitoring.models.filters.TranslatedFilter[source]
Bases:
QueryFilter
Translated filters use the SMC internal name alias and builds expressions to make more complex queries.
Example of using built in filter methods:
query = LogQuery(fetch_size=50) query.format.timezone('CST') query.format.field_format('name') translated_filter = query.add_translated_filter() translated_filter.within_ipv4_network('$Dst', ['192.168.4.0/24']) translated_filter.within_ipv4_range('$Src', ['1.1.1.1-192.168.1.254']) translated_filter.exact_ipv4_match('$Src', ['172.18.1.152', '192.168.4.84'])
- exact_ipv4_match(field, values)[source]
An exact IPv4 address match on relevant address fields.
- Parameters:
field (str) – name of field to filter on. Taken from ‘Show Filter Expression’ within the Management Client.
values (list) – value/s to add. If more than a single value is provided, the query is modified to use UNION vs. ==
complex (bool) – A complex filter is one which requires AND’ing or OR’ing values. Set to return the filter before committing.
- within_ipv4_network(field, values)[source]
This filter adds specified networks to a filter to check for inclusion.
Values
Values are used to provide searchable input for filters.
Each value format is specific to the data type added to the filter. For example,
an IPValue specifies IP’s or network values that can be added to a filter from
smc_monitoring.models.filters
.
Each constructor can be initialized in the following ways:
Single value:
IPValue('1.1.1.1')
Multiple values:
IPValue('1.1.1.1', '2.2.2.2')
As a list of values:
i = ['1.1.1.1', '3.3.3.3']
IPValue(*i)
The value
attribute of each Value
stores the query string as a list
that is absorbed by the filter.
- class smc_monitoring.models.values.ConstantValue(*constants)[source]
Bases:
Value
Constant values can be used for log field values. For example, specifying a filter by Action can be simplified by specifying the constant for the action value. Constant values are not used for log field names (use FieldValue instead).
Searching for all actions of discard and block:
query = LogQuery(fetch_size=50) query.add_in_filter( FieldValue(LogField.ACTION), [ConstantValue(Actions.DISCARD, Actions.BLOCK)])
- class smc_monitoring.models.values.ElementValue(*elements)[source]
Bases:
Value
Element Values are used when creating a filter for an element already defined in the Mangement Server database. The element can be referenced by it’s type.
Search for a host element ‘kali’ in the ‘source’ log field:
query = LogQuery(fetch_size=50) query.add_in_filter( FieldValue(LogField.SRC), [ElementValue(Host('kali'))])
Note
Using elements expands the search to potentially include a broader range of data. For example, a host can have multiple IP addresses, both ipv4 and ipv6.
- class smc_monitoring.models.values.FieldValue(*fields)[source]
Bases:
Value
FieldValue specifies a log field filter by either constant ID or name. The field name field is the internal name representation for the Management Client. To find a given field name, in the Logs view of the Management Client, drag a field into the filter window, right click and select “Show Filter Expression”.
Using field value as filter for InFilter type:
query = LogQuery(fetch_size=50) query.add_in_filter( FieldValue(LogField.SRC), [IPValue('192.168.4.84')])
Note
If using constant values, consult
smc.monitoring.constants.LogField
for valid attributes.
- class smc_monitoring.models.values.IPValue(*addresses)[source]
Bases:
Value
IP Values specify IP addresses used for searching.
Search for IP address in source and dest fields:
query = LogQuery(fetch_size=50) query.add_in_filter( IPValue('192.168.4.84'), [FieldValue(LogField.SRC, LogField.DST)])
- class smc_monitoring.models.values.NumberValue(*values)[source]
Bases:
Value
Number value match.
Search for port in source fields:
query = LogQuery(fetch_size=10) query.add_in_filter(FieldValue(LogField.SPORT), [NumberValue(7000, 7001)])
- class smc_monitoring.models.values.ServiceValue(*services)[source]
Bases:
Value
Service Values allow searches on the service field. When specifying the service value, specify as <protocol/port>. For example, ‘TCP/80’, ‘UDP/53’. For ICMP, specify as ICMP/Type/Code (Code is optional).
Search for any services with TCP port 80 and UDP port 53:
query = LogQuery(fetch_size=50) query.add_in_filter( FieldValue(LogField.SERVICE), [ServiceValue('TCP/80', 'UDP/53')])
- class smc_monitoring.models.values.StringValue(*values)[source]
Bases:
Value
String value match. Note that string matching can only be done on log fields that are of type string (no type conversions are done on non-string types). String matches are also exact.
Find all audits accessing URL play.googleapis.com:
query = LogQuery(fetch_size=50) query.add_in_filter( FieldValue(LogField.HTTPREQUESTHOST),[StringValue('play.googleapis.com')])
- Parameters:
value (list) – string to match
- class smc_monitoring.models.values.TranslatedValue(value)[source]
Bases:
Value
Internal SMC filter format value match. To use with “translated” filter
Search for port in destination fields:
query = LogQuery(fetch_size=10) query_filter = QueryFilter(“translated”) query_filter.update_filter(TranslatedValue(“$Dport == 22 OR $Dport == 25”).value) query.update_filter(query_filter)
- Parameters:
value (str) – specifies a string in the internal SMC filter format.
Formats
Field formats represent a way to control the format of the returned data. By modifying a field format, you can control field level settings such as wther to resolve IP’s via DNS, how to display field names and values and which fields to return in the query.
Each log format will return a different view type The most common and default
for all queries is the TextFormat
using a ‘pretty’ field format
which is what you will see from the column data and values if using the
Logs view of the Management Client.
Return only a specific set of fields by id’s:
query = LogQuery(fetch_size=5)
query.format.field_ids([
LogField.TIMESTAMP, LogField.NODEID, LogField.SRC,
LogField.DST, LogField.PROTOCOL, LogField.ACTION])
Return only a specific set of fields by name:
query = LogQuery(fetch_size=5)
query.format.field_names(['Src', 'Dst'])
Note
If both field_ids and field_names are provided, they will be merged.
- class smc_monitoring.models.formats.CombinedFormat(**kw)[source]
Bases:
object
CombinedFormat provides a way to specify different field resolvers based on field name or ID. Keyword arguments provided will define a unique key that represents the format object and value is the format object itself.
For example, using a combined filter to resolve the TIMESTMAP field in text format, but source and destination fields in detailed format:
text = TextFormat() text.field_ids([LogField.TIMESTAMP]) detailed = DetailedFormat() detailed.field_ids([LogField.SRC, LogField.DST]) combined = CombinedFormat(tformat=text, dformat=detailed) query = LogQuery(fetch_size=1, format=combined)
After executing the query, the raw record results will be formatted as a list of dict’s, which each record having a dict key equal to keyword argument input provided:
[{'dformat': {'Src Addr': '10.0.0.1', 'Dst Addr': '224.0.0.1'}, 'tformat': {'Creation Time': '2017-08-05 14:12:44'}}, {'dformat': {'Src Addr': '10.0.0.1', 'Dst Addr': '224.0.0.1'}, 'tformat': {'Creation Time': '2017-08-05 14:12:44'}}] ...
The results can then be parsed and used to provide custom views as necessary.
- Parameters:
kw – key word arguments should use an identifier key that will be present in the results, and a value which is a format object type in
smc.monitoring.formats
.
- class smc_monitoring.models.formats.DetailedFormat(field_format='pretty', **kw)[source]
Bases:
TextFormat
Detailed format does not do a Log value conversion as the TextFormat would, however does provide a field map in the first payload with characteristics of the fields in the return data. This might be a useful format to obtain conversion ID’s for specific fields or debugging.
- class smc_monitoring.models.formats.FormatFieldMixin[source]
Bases:
object
Format field methods for modifying behavior of a query.
- field_format(name)[source]
Specify how the field name are printed in the response.
- Parameters:
id (str) – as integer IDs from constants found in
smc_monitoring.models.constants.LogField
name (str) – as internal SMC names
pretty (str) – pretty printed as you would see in the Management Client
- field_ids(ids)[source]
Add filter to show only fields with given field ID’s. Field ID’s can be mapped to the LogField constants in
smc_monitoring.models.constants.LogField
Note
Set the return display mode for the Log field name by using
field_format()
. The display valuename
will match the name of the LogField constant.
- field_names(names)[source]
Show only fields with given name. The
name
is the name for the log field in the Management Client. The simplest way to obtain the name for a log field is from the Logs view of the Management Client. Use the Query pane in the Logs view of the Management Client to drag a column filter and select “Show Filter Expression”- ..note:: The log field name is case sensitive and is typically
using camelcase notation.
- class smc_monitoring.models.formats.RawFormat(field_format='pretty')[source]
Bases:
FormatFieldMixin
Raw format is an abbreviated version of the detailed format. Fewer fields are provided and resolution of field values is not done.
- class smc_monitoring.models.formats.TextFormat(field_format='pretty', **kw)[source]
Bases:
FormatFieldMixin
Text format with ‘pretty’ field formatting uses the same display to what you would see from the native SMC Log Viewer.
Keyword arguments can optionally be provided to set ‘resolving’ fields during instance creation, or they can be set on the instance afterwards by calling
set_resolving()
.- set_resolving(**kw)[source]
Certain log fields can be individually resolved. Use this method to set these fields. Valid keyword arguments:
- Parameters:
timezone (str) – string value to set timezone for audits
time_show_zone (bool) – show the time zone in the audit.
time_show_millis (bool) – show timezone in milliseconds
keys (bool) – resolve log field keys
ip_elements (bool) – resolve IP’s to SMC elements
ip_dns (bool) – resolve IP addresses using DNS
ip_locations (bool) – resolve locations
Constants
Constants used within smc_monitoring.models.values.Value
values to simplify
referencing log viewer data.
- class smc_monitoring.models.constants.Actions[source]
Bases:
object
Rule Actions
- ALLOW = 1
Allowed
- BLOCK = 13
Block
- DISCARD = 0
Discard
- DISCARD_PASSIVE = 4
Silent discard
- PERMIT = 11
Permit the connection
- REFUSE = 2
Reset
- TERMINATE = 9
Terminate
- TERMINATE_FAILED = 10
Failed terminating connection
- TERMINATE_PASSIVE = 8
Silent terminate
- TERMINATE_RESET = 12
Reset the connection
- class smc_monitoring.models.constants.Alerts[source]
Bases:
object
Alert actions
- CRITICAL = 10
Critical alert
- HIGH = 5
High alert
- INFO = 1
Info alert
- LOW = 3
Low alert
- class smc_monitoring.models.constants.DataType[source]
Bases:
object
Query by type of logs. This identifies which log types you are interested in filtering by, i.e. Audit, FW Logs, Third_Party, etc. Equivalent to the Query dropdown in the Logs view of the Management Client
- class smc_monitoring.models.constants.LogField[source]
Bases:
object
Log field constants can be referenced when creating filters such as Field Values. i.e. FieldValue(LogField.SRC). Each constant name is identical to the value when using the field format type of ‘name’ (with exception that the constant names are in upper case).
- ACCELAPSED = 104
Elapsed time of connection in seconds
- ACCRXBYTES = 106
Number of bytes received during connection
- ACCRXPACKETS = 139
Number of packets received during connection
- ACCTXBYTES = 105
Number of bytes sent during connection
- ACCTXPACKETS = 138
Number of packets sent during connection
- ACK = 29
Acknowledged Alert
- ACTION = 14
Connection action
- ALERT = 25
Type of alert
- ALERTCOUNT = 603
Alert count
- ALERTERTRACE = 600
Alerter trace (events) information (datatype:4)
- ALERTSEVERITY = 602
Severity of situation
- ALERTSTATUS = 604
Alert status
- ALLOWEDDATATAG = 482
Allowed data type tag
- APPLICATION = 800
Application
- APPLICATIONCOMBINATIONFLAGS = 54
Anomaly information of certain combination of network application and client application.
- APPLICATIONDETAIL = 801
Application Detail
- APPLICATIONUSAGE = 52
The type of the application that caused sending this event.
- ASPAMEMAILMESSAGEID = 155
Email message-ID
- ASPAMEMAILSCORE = 153
Email score value
- ASPAMEMAILSUBJECT = 152
Email subject
- ASPAMRECEIVEREMAIL = 151
Receiver email address
- ASPAMSENDEREMAIL = 150
Sender email address
- ASPAMSENDERMTA = 154
Sender Message Transfer Agent IP address
- AUTHENTICATIONCOUNTER = 850
Authentication counters
- AUTHMETHOD = 133
Authentication Method element
- AUTHNAME = 108
User name of authorized user
- AUTHRULEID = 107
The rule number of the rule that led to the log creation
- BALANCINGPROBING = 397
BALANCING_PROBING
- BALANCINGSELECTION = 392
BALANCING_SELECTION
- BLACKLISTENTRYDESTINATIONIP = 120
Blacklist entry destination IP address
- BLACKLISTENTRYDESTINATIONIPMASK = 121
Blacklist entry destination IP address mask
- BLACKLISTENTRYDESTINATIONPORT = 125
Blacklist entry destination port
- BLACKLISTENTRYDESTINATIONPORTRANGE = 126
Blacklist entry destination port range end
- BLACKLISTENTRYDURATION = 127
Blacklist entry duration
- BLACKLISTENTRYID = 117
None
- BLACKLISTENTRYPROTOCOL = 122
Blacklist entry IP protocol
- BLACKLISTENTRYSOURCEIP = 118
Blacklist entry source IP address
- BLACKLISTENTRYSOURCEIPMASK = 119
Blacklist entry source IP address mask
- BLACKLISTENTRYSOURCEIPPREFIXLEN = 172
Blacklist entry source IP address prefix length
- BLACKLISTENTRYSOURCEPORT = 123
Blacklist entry source port
- BLACKLISTENTRYSOURCEPORTRANGE = 124
Blacklist entry source port range end
- BLACKLISTER = 128
Blacklister
- BLOCK_LISTENTRYDESTINATIONIP = 120
block_list entry destination IP address
- BLOCK_LISTENTRYDESTINATIONIPMASK = 121
block_list entry destination IP address mask
- BLOCK_LISTENTRYDESTINATIONPORT = 125
block_list entry destination port
- BLOCK_LISTENTRYDESTINATIONPORTRANGE = 126
block_list entry destination port range end
- BLOCK_LISTENTRYDURATION = 127
block_list entry duration
- BLOCK_LISTENTRYID = 117
None
- BLOCK_LISTENTRYPROTOCOL = 122
block_list entry IP protocol
- BLOCK_LISTENTRYSOURCEIP = 118
block_list entry source IP address
- BLOCK_LISTENTRYSOURCEIPMASK = 119
block_list entry source IP address mask
- BLOCK_LISTENTRYSOURCEIPPREFIXLEN = 172
block_list entry source IP address prefix length
- BLOCK_LISTENTRYSOURCEPORT = 123
block_list entry source port
- BLOCK_LISTENTRYSOURCEPORTRANGE = 124
block_list entry source port range end
- BLOCK_LISTER = 128
block_lister
- CIPHERALG = 536
Cipher algorithm
- CLIENTIPADDRESS = 403
Address of client causing event
- COMPID = 3
The identifier of the creator of the log entry.
- CONNDIRECTION = 310
Connection direction
- CONNECTEDMACADDR = 447
Connected MAC addresses
- CONNECTIVITY = 306
Connectivity
- CONNSTATUS = 309
Connection status
- CONNTYPE = 308
Connection type
- CONTAINEDDATATAG = 483
Contained data type tag
- CONTROLCOMMANDID = 28
None
- DATATAG = 481
Data type tag
- DATATAGS = 485
Data tags concerning the record
- DATATYPE = 34
Data type
- DHCPLEASEEXPIRES = 528
DHCP_LEASE_EXPIRES
- DHCPLEASEGW = 529
DHCP_LEASE_GW
- DHCPLEASEIP = 530
DHCP_LEASE_IP
- DHCPLEASENETMASK = 531
DHCP_LEASE_NETMASK
- DHCPLEASEPREFIXLEN = 498
DHCP_LEASE_PREFIXLEN
- DHCPLEASERECEIVED = 532
DHCP_LEASE_RECEIVED
- DHCPLEASES = 527
DHCP_LEASES
- DPD = 543
Dead Peer Detection
- DPORT = 10
Connection destination protocol port
- DSCPMARK = 130
DSCP Mark
- DST = 8
Connection destination IP address
- DSTADDRS = 20008
Destination addresses
- DSTIF = 13
Destination interface of firewall
- DSTIPRANGE = 526
Destination IP Range
- DSTVLAN = 113
Destination VLAN
- DSTZONE = 47
Connection destination interface zone
- ELEMENTDOMAIN = 415
Administrative Domain of Associated Element
- ENDPOINT = 504
Local VPN end point
- ENTERPRISEOID = 493
Enterprise OID
- EVENT = 6
Logged event
- EVENTADDRESS = 705
Notification destination
- EVENTINFO = 701
Description for event
- EVENTLOGID = 702
Data Identifier of the alert
- EVENTTIME = 700
Time stamp of the alert
- EVENTTYPE = 703
Type of event
- EVENTUSER = 704
User who executed the action
- EXPIRATIONTIME = 534
VPN SA expiration time
- FACILITY = 22
Firewall subsystem
- FILETYPECOMPAT = 56
The type of the file that caused sending this event.
- FLAG = 114
None
- FPCACHED = 57
Fingerprint match came from fingerprinting cache.
- FW100INTERFACE = 431
FW100 Interface
- FW100TRAFFICCOUNTERS = 430
Fw100 Traffic counters
- FWACCEPTEDBYTES = 326
FW_ACCEPTED_BYTES
- FWACCEPTEDPACKETS = 327
FW_ACCEPTED_PACKETS
- FWACCOUNTEDBYTES = 336
FW_ACCOUNTED_BYTES
- FWACCOUNTEDPACKETS = 337
FW_ACCOUNTED_PACKETS
- FWADSLRXBYTES = 417
FW_ADSL_RX_BYTES
- FWADSLTXBYTES = 416
FW_ADSL_TX_BYTES
- FWDECRYPTEDBYTES = 332
FW_DECRYPTED_BYTES
- FWDECRYPTEDPACKETS = 333
FW_DECRYPTED_PACKETS
- FWDROPPEDBYTES = 328
FW_DROPPED_BYTES
- FWDROPPEDPACKETS = 329
FW_DROPPED_PACKETS
- FWENCRYPTEDBYTES = 330
FW_ENCRYPTED_BYTES
- FWENCRYPTEDPACKETS = 331
FW_ENCRYPTED_PACKETS
- FWFORWARDEDBYTES = 419
FW_FORWARDED_BYTES
- FWFORWARDEDPACKETS = 418
FW_FORWARDED_PACKETS
- FWINTERFACEKEY = 340
FW_INTERFACE_KEY
- FWNATTEDBYTES = 334
FW_NATTED_BYTES
- FWNATTEDPACKETS = 335
FW_NATTED_PACKETS
- FWRECEIVEDBYTES = 322
FW_RECEIVED_BYTES
- FWRECEIVEDPACKETS = 323
FW_RECEIVED_PACKETS
- FWSENTBYTES = 324
FW_SENT_BYTES
- FWSENTPACKETS = 325
FW_SENT_PACKETS
- FWTRAFFIC = 342
FW Traffic
- FWTRAFFICACCOUNTEDBYTES = 352
Accounted Bytes
- FWTRAFFICACCOUNTEDPACKETS = 346
Accounted Packets
- FWTRAFFICALLOWEDBYTES = 349
Allowed Bytes
- FWTRAFFICALLOWEDPACKETS = 343
Allowed Packets
- FWTRAFFICDISCARDEDBYTES = 350
Discarded Bytes
- FWTRAFFICDISCARDEDPACKETS = 344
Discarded Packets
- FWTRAFFICENCRYPTEDBYTES = 354
Encrypted Bytes
- FWTRAFFICENCRYPTEDPACKETS = 348
Encrypted Packets
- FWTRAFFICLOGGEDBYTES = 351
Logged Bytes
- FWTRAFFICLOGGEDPACKETS = 345
Logged Packets
- FWTRAFFICNATTEDBYTES = 353
Natted Bytes
- FWTRAFFICNATTEDPACKETS = 347
Natted Packets
- GENERICTRAPTYPE = 494
Generic Trap Type
- HASHALG = 538
Hash Algorithm
- HITS = 48
HITS
- HTTPREQUESTHOST = 1586
HTTP request host
- ICMPCODE = 101
ICMP code attribute
- ICMPID = 102
ICMP identifier
- ICMPTYPE = 100
ICMP type attribute
- IKEDHGROUP = 901
Diffie-Hellman Group
- IKELOCALID = 540
Local IKE ID
- IKEREMOTEID = 541
Remote IKE ID
- IKEV1MODE = 542
IKEv1 negotiation mode
- INCIDENTCASE = 411
Incident Case
- INFOMSG = 19
Information Message
- INTERFACE = 35
Interface
- IPCOMPRESSION = 546
IP Compression
- IPSAPPID = 134
Network application detected in the connection
- IPSECSSPI = 103
Inbound IPsec SPI value (hexadecimal)
- LOGID = 2
Data Identifier
- LOGIFTOPDESTINATIONIPADDRS = 446
Amount of traffic flowing to the most used destination IP addresses per logical interface
- LOGIFTOPSOURCEIPADDRS = 445
Amount of traffic originating from the most used source IP addresses per logical interface
- LOGSEVERITY = 805
Severity
- LONGMSG = 601
Long field description of alert
- MACALG = 537
MAC Algorithm
- MESSAGEID = 804
Message Id
- NATBALANCEID = 393
NAT_BALANCE_ID
- NATDPORT = 18
Translated packet destination port
- NATDST = 16
Translated packet destination IP address
- NATMAPID = 394
NAT_MAP_ID
- NATRULEID = 21
The rule number of the rule that led to the log creation
- NATSPORT = 17
Translated packet source protocol port
- NATSRC = 15
Translated packet source IP address
- NATT = 544
NAT Traversal
- NEGOTIATIONROLE = 539
SA Negotiation Role
- NEIGHBORINTERFACE = 733
Interface
- NEIGHBORL2DATA = 736
Mac address
- NEIGHBORL3DATA = 735
IP Address
- NEIGHBORPROTOCOL = 734
Protocol
- NEIGHBORSTATE = 737
State
- NODECAPACITY = 321
Capacity
- NODECONFIGURATION = 304
Current configuration
- NODECONFIGURATIONTIMESTAMP = 305
Configuration upload time
- NODEDYNUP = 303
Update package level
- NODEHWSTATUS = 315
Node hardware status
- NODEID = 4
Firewall or server node that passes this information
- NODELOAD = 320
Node load
- NODESTATUS = 300
Node status
- NODEVERSION = 301
Node version
- NONCONTAINEDDATATAG = 484
Non-contained data type tag
- NUMALERTRESPONSES = 365
Number of alert responses performed by this engine
- NUMBLACKLISTRESPONSES = 369
Number of blacklist responses performed by this engine
- NUMBLOCK_LISTRESPONSES = 369
Number of block_list responses performed by this engine
- NUMBYTESRECEIVED = 12201
Number of bytes received, used for VPN
- NUMBYTESSENT = 12200
Number of bytes sent, used for VPN
- NUMDISCARDRESPONSES = 368
Number of discard responses performed by this engine
- NUMLOGEVENTS = 363
Number of log events
- NUMLOGRESPONSES = 364
Number of log responses performed by this engine
- NUMPACKETSRECEIVED = 549
Number of packets received
- NUMPACKETSSENT = 548
Number of packets sent
- NUMRECORDRESPONSES = 366
Number of record responses performed by this engine
- NUMRESETRESPONSES = 367
Number of reset responses performed by this engine
- OBJECTDN = 410
User and Group Information
- OBJECTKEY = 409
Element Id
- OBJECTNAME = 407
Elements being manipulated
- OBJECTTYPE = 408
Element Type
- ORIGINNAME = 400
Name of component producing event
- OUTBOUNDSPI = 533
Outbound IPsec SPI value (hexadecimal)
- PASSEDBYTES = 388
PASSED_BYTES
- PEERCOMPONENTID = 307
Peer component id
- PEERENDPOINT = 506
Peer VPN end point
- PEERSECURITYGATEWAY = 505
Peer VPN gateway
- PFSDHGROUP = 547
PFS Diffie-Hellman Group
- PHASE1FAIL = 511
IKE_PHASE1_FAIL
- PHASE1SUCC = 510
IKE_PHASE1_SUCC
- PHASE2FAIL = 513
IKE_PHASE2_FAIL
- PHASE2SUCC = 512
IKE_PHASE2_SUCC
- POTENTIALLYDUPLICATERESPONSE = 170
Potentially duplicate correlation response
- PROBEFAIL = 500
PROBE_FAIL
- PROBEOK = 399
PROBE_OK
- PROTOCOL = 11
IP protocol
- QOSCLASS = 129
QoS Class
- QOSPRIORITY = 131
QoS Priority
- RADIUSACCOUNTINGTYPE = 851
Radius Accounting Type
- RECEIVEDLOGEVENTS = 361
RECEIVED_LOG_EVENTS
- RECEPTIONTIME = 24
Reception Time on the log Server
- RESOURCE = 806
Resource
- RESULT = 405
Result state
- RETSRCIF = 49
Return source interface of the connection
- ROUTEBGPPATH = 167
Active BGP path
- ROUTEDISTANCE = 162
Relative distance for route validation
- ROUTEGATEWAY = 164
IP address of the gateway for the route
- ROUTEMETRIC = 163
Protocol specific metric value
- ROUTENETMASK = 161
Netmask address of the network
- ROUTENETWORK = 160
Network address of the network
- ROUTEOSPFLSATYPE = 166
Type of OSPF LSA’s
- ROUTETYPE = 165
Type of route
- RTT = 109
Round trip time of connection establishing
- RULECOUNTERS = 412
RULE_COUNTERS
- RULEHITS = 413
RULE_HITS
- RULEID = 20
Rule tag value of acceptance rule
- RWPHTTPREFERRER = 832
HTTP Referrer
- RWPHTTPUSERAGENT = 830
HTTP User Agent
- RWPSERVICENAME = 831
SSL VPN Portal Service Name
- SAAUTHALG = 520
SA_AUTH_ALG
- SABUNDLE = 514
SA_BUNDLE
- SACIPHERALG = 518
SA_CIPHER_ALG
- SACLASS = 535
SA Type
- SACOMPRESSIONALG = 519
SA_COMPRESSION_ALG
- SAEXPIREHARDLIMIT = 524
SA_EXPIRE_HARDLIMIT
- SAEXPIRESOFTLIMIT = 523
SA_EXPIRE_SOFTLIMIT
- SAINCOMING = 517
SA_INCOMING
- SAKBHARDLIMIT = 522
SA_KB_HARDLIMIT
- SAKBSOFTLIMIT = 521
SA_KB_SOFTLIMIT
- SARESPONDER = 516
SA_RESPONDER
- SATYPE = 515
SA_TYPE
- SECURITYGATEWAY = 502
VPN gateway
- SELECTEDCACHE = 396
SELECTED_CACHE
- SELECTEDRTT = 395
SELECTED_RTT
- SENDER = 5
None
- SENDERDOMAIN = 38
Administrative Domain of Event Sender
- SENDERTYPE = 31
Sender type
- SENSORALLOWEDINSPECTEDTCPCONNECTIONS = 437
- SENSORALLOWEDINSPECTEDUDPCONNECTIONS = 438
- SENSORALLOWEDUNINSPECTEDTCPCONNECTIONS = 439
- SENSORALLOWEDUNINSPECTEDUDPCONNECTIONS = 440
- SENSORDISCARDEDTCPCONNECTIONS = 441
- SENSORDISCARDEDUDPCONNECTIONS = 442
- SENSORINSPECTEDBYTES = 357
Bytes inspected by sensor
- SENSORINSPECTEDPACKETS = 358
Packets inspected by sensor
- SENSORINTERFACEKEY = 370
Sensor interface key
- SENSORLOSTBYTES = 359
Bytes lost in sensor
- SENSORLOSTPACKETS = 360
Packets lost in sensor
- SENSORPROCESSEDBYTES = 355
Bytes processed by sensor
- SENSORPROCESSEDPACKETS = 356
Packets processed by sensor
- SENSORRECEIVEDBYTES = 338
Bytes received by sensor
- SENSORRECEIVEDPACKETS = 339
Packets received by sensor
- SENSORTRAFFIC = 372
Sensor traffic
- SENSORTRAFFICCLOSEDTCPCONNECTIONS = 383
Closed TCP Connections
- SENSORTRAFFICINSPECTEDPACKETS = 376
Inspected Packets
- SENSORTRAFFICLOSTPACKETS = 375
Lost Packets
- SENSORTRAFFICNEWTCPCONNECTIONS = 381
New TCP Connections
- SENSORTRAFFICNUMBEROFALERTS = 380
Number of Alerts
- SENSORTRAFFICOKCONNECTIONS = 378
OK Connections
- SENSORTRAFFICPROCESSEDBYTES = 374
Processed Bytes
- SENSORTRAFFICPROCESSEDPACKETS = 373
Processed Packets
- SENSORTRAFFICSTATSOFPACKETS = 377
Stats Of Packets
- SENSORTRAFFICSUSPICIOUSCONNECTIONS = 379
Suspicious Connections
- SENSORTRAFFICTCPHANDSHAKES = 382
TCP Handshakes
- SENSORTRAFFICTCPTIMEOUTS = 384
TCP Timeouts
- SENTLOGEVENTS = 362
SENT_LOG_EVENTS
- SERVICEKEY = 132
Service primary key, used in service resolving
- SESSIONDOMAIN = 414
Administrative Domain of Login Session
- SESSIONEVENT = 302
Session monitoring event code (1 = new, 2 = update, 3 = remove, 4 = all sessions sent)
- SESSIONID = 802
Id of the User Session
- SFPINGRESS = 900
SFP_INGRESS
- SHAPINGCLASS = 386
SHAPING_CLASS
- SHAPINGGUARANTEE = 389
SHAPING_GUARANTEE
- SHAPINGLIMIT = 390
SHAPING_LIMIT
- SHAPINGPRIORITY = 391
SHAPING_PRIORITY
- SITCATEGORY = 37
The type of the situation that caused sending this event.
- SITUATION = 1000
The identifier of the situation that caused sending this event.
- SNMPRETSRCIF = 51
SNMP index of return source interface
- SNMPSRCIF = 50
SNMP index of source interface
- SNMPTRAPMAP = 490
SNMP Trap
- SNMPTRAPOID = 491
SNMP Trap OID
- SNMPTRAPVALUE = 492
SNMP Trap Value
- SPORT = 9
Connection source protocol port
- SRC = 7
Connection source IP address
- SRCADDRESS = 398
SRC_ADDRESS
- SRCADDRS = 20007
Source addresses
- SRCIF = 12
Source interface of firewall
- SRCIPRANGE = 525
Source IP Range
- SRCVLAN = 112
Source VLAN
- SRCZONE = 46
Connection source interface zone
- SRVHELPERID = 110
Protocol agent identification
- SSLVPNSESSIONMONID = 811
Id of the User Session
- SSLVPNSESSIONMONRECEIVED = 809
Node’s local time when the SSL VPN session was created
- SSLVPNSESSIONMONTIMEOUT = 810
Node’s local time when the SSL VPN session will time-out
- SSLVPNSESSIONTYPETYPE = 808
SSL VPN session client type
- STATE = 116
Connection state in connection monitoring
- STATUSTYPE = 311
Status type
- STORAGESERVERID = 30
Storage Server
- SYSLOGTYPE = 111
Syslog message type
- TAGINFO = 480
Type tags
- TCPDUMPSTATUS = 318
TCPDump Monitoring Status
- TCPENCAPSULATION = 545
TCP Encapsulation
- TIMEOUT = 115
Connection timeout in connection monitoring
- TIMESTAMP = 1
Time of creating the event record.
- TLSALERTDESCRIPTION = 45
TLS/SSL Alert Message Description
- TLSALERTLEVEL = 44
TLS/SSL Alert Message Alert Level
- TLSCERTIFICATEVERIFYERRORCODE = 39
TLS/SSL Certificate verify error code
- TLSCIPHERSUITE = 42
TLS/SSL cipher suite
- TLSCOMPRESSIONMETHOD = 43
TLS/SSL compression method
- TLSDETECTED = 136
The connection uses SSL/TLS protocol.
- TLSDOMAIN = 40
Domain name field in SSL/TLS certificate
- TLSPROTOCOLVERSION = 41
TLS/SSL protocol version
- TOTALBYTES = 387
TOTAL_BYTES
- TPACCEPTEDBYTES = 465
TP_ACCEPTED_BYTES
- TPACCEPTEDPACKETS = 466
TP_ACCEPTED_PACKETS
- TPDROPPEDBYTES = 467
TP_DROPPED_BYTES
- TPDROPPEDPACKETS = 468
TP_DROPPED_PACKETS
- TPMEMUSAGE = 470
Third party memory usage
- TPNODELOAD = 469
Third party device load
- TPRECEIVEDBYTES = 461
TP_RECEIVED_BYTES
- TPRECEIVEDPACKETS = 462
TP_RECEIVED_PACKETS
- TPSENTBYTES = 463
TP_SENT_BYTES
- TPSENTPACKETS = 464
TP_SENT_PACKETS
- TPTRAFFICCOUNTERS = 460
Third party traffic counters
- TRAFFICCOUNTERS = 319
Traffic counters
- TRAFFICSHAPING = 385
TRAFFIC_SHAPING
- TRANSIENT = 26
None
- TUNNELINGLEVEL = 95
Number of tunneling protocol layers encapsulating this protocol layer
- TYPE = 23
Log event severity type
- TYPEDESCRIPTION = 404
Description of the event
- URLCATEGORYGROUP = 53
The type of the URL that caused sending this event.
- URLCATEGORYRISK = 55
The risk of the URL that caused sending this event.
- USERNAME = 3001
Username if present
- USERORIGINATOR = 401
Administrator causing event
- USERROLE = 402
Roles of Administrator causing event
- VPNBYTESRECEIVED = 509
VPN_BYTES_RECEIVED
- VPNBYTESSENT = 508
VPN_BYTES_SENT
- VPNID = 501
Desination VPN
- VPNSRCID = 499
Source VPN
- VPNSTATISTICS = 507
VPN_STATISTICS
- VPNSTATUS = 503
VPN_STATUS
- VPNTYPE = 611
VPN_TYPE
- VULNERABILITYREFERENCES = 20000
Generated from situation and original situation.
- WIRELESSCHANNEL = 448
Wireless Access Point’s channel
- WIRELESSCONNECTIONS = 436
Number of wireless connections
- WIRELESSMONITORING = 432
Wireless Monitoring
- WIRELESSSECURITY = 435
Wireless Security mode
- WIRELESSSSID = 433
Wireless SSID
- WIRELESSSTATUS = 434
Wireless Status
- ZIPEXPORTFILE = 420
Snapshot of element being manipulated
Formatters
Custom formats used to return data in different formats. These are used from the query itself when calling the fetch_as_format() method. For example, returning a LogQuery as a table:
query = LogQuery(fetch_size=200)
for log in query.fetch_batch(): # Default is TableFormat
print(log)
As CSV:
query = LogQuery(fetch_size=200)
for log in query.fetch_batch(CSVFormat):
print(log)
Each format also allows the ability to customize the fields that should be
in the output. By default, each query type in smc_monitoring.monitors
will have a class attribute field_ids
which specify the default fields.
These can be customized by modifying the query.format.field_ids([….])
parameter.
For example, modifying a routing query to return only destination interface and the route network:
query = RoutingQuery('sg_vm')
query.format.field_ids([LogField.DSTIF, LogField.ROUTENETWORK])
for log in query.fetch_batch():
...
The same field_id customization applies to all query types.
A simple way to view results is to use a RawDictFormat:
query = LogQuery(fetch_size=3)
query.format.field_names(['Src', 'Dst'])
for record in query.fetch_batch(RawDictFormat):
...
It is also possible to provide your own formatter. At a minimum you must
provide a method called formatted
in your class. The custom class should
extend _Header
to support custom field_ids within the query.
Note
Constants are defined in smc_monitoring.models.constants
.
Although there are many field values, not all field values will return
results for every query. It is sometimes useful to log in to the Management
Client to verify available fields.
- class smc_monitoring.models.formatters.CSVFormat(query)[source]
Bases:
_Header
Return the results in CSV format. The first line will be a comma separated string with the field header. This is an iterable that will return results in batches of 200 (max) per iteration.
- class smc_monitoring.models.formatters.ElementFormat(query)[source]
Bases:
RawDictFormat
Return the data as a list in Element format.
- exception smc_monitoring.models.formatters.InvalidFieldFormat[source]
Bases:
Exception
If using a complex format type such as combined, formatters are not supported. These specialized formats must be returned in raw dict format as they’ve been customized to return the data in a specific way.
- class smc_monitoring.models.formatters.RawDictFormat(query)[source]
Bases:
object
Return the data as a list in raw dict format. The results are not filtered with exception of the returned fields based on field_id filters. This is a convenience format for consistency, although you can also call the
smc_monitoring.models.query.Query.fetch_raw
method to get the same data.
- class smc_monitoring.models.formatters.TableFormat(query)[source]
Bases:
_Header
Return the data in a table format. The field_id values will be used for the table header. Spacing will be calculated for each batch of results to align the table. The base spacing is determined by the header width, but adjusted wider if the data returned is wider. Anytime there is an adjustment to the width, a new table header will also be printed to visually realign. The query will return a max of 200 batch results per iteration.
Note
Table alignment will likely not be exact between batches as width is calculated per batch.
TimeRanges
Time formats are optionally used in a LogQuery to specify custom ranges for which to search ‘stored’ log events.
When adding a time format to a query, the start_time
and end_time
values
need to be in milliseconds. The engine logs are stored in UTC time but in order
to display the client side dates properly, you should set a timezone on the
query.
There are helper methods to simplify retrieving for last_XXX period of time as well as custom range formats.
Set up a query with a time format:
query = LogQuery(fetch_size=50)
query.format.timezone('Europe/Helsinki')
query.time_range.last_five_minutes()
See also
custom_range()
for more examples on creating
custom time range formats.
- class smc_monitoring.models.calendar.TimeFormat(start_ms=0, end_ms=0)[source]
Bases:
object
Construct a time format to control the start and end times for a query. If unspecified, results will be limited by the fetch size quantity only. Helper methods are provided to simplify adding time based filters once the instance is constructed.
- Parameters:
- custom_range(start_time, end_time=None)[source]
Provide a custom range for the search query. Start time and end time are expected to be naive
datetime
objects converted to milliseconds. When submitting the query, it is strongly recommended to set the timezone matching the local client making the query.Example of finding all records on 9/2/2017 from 06:25:30 to 06:26:30 in the local time zone CST:
dt_start = datetime(2017, 9, 2, 6, 25, 30, 0) dt_end = datetime(2017, 9, 2, 6, 26, 30, 0) query = LogQuery() query.format.timezone('CST') query.time_range.custom_range( datetime_to_ms(dt_start), datetime_to_ms(dt_end)) for record in query.fetch_batch(): print(record)
Last two minutes from current (py2):
now = datetime.now() start_time = int((now - timedelta(minutes=2)).strftime('%s'))*1000
Specific start time (py2):
p2time = datetime.strptime("1.8.2017 08:26:42,76", "%d.%m.%Y %H:%M:%S,%f").strftime('%s') p2time = int(s)*1000
Specific start time (py3):
p3time = datetime.strptime("1.8.2017 08:40:42,76", "%d.%m.%Y %H:%M:%S,%f") p3time.timestamp() * 1000
- property end_time
Return the end time in datetime format. Will return 0 if end time is not specified.
- Return type:
datetime
- property start_time
Return the start time in datetime format. Will return 0 if start time is not specified.
- Return type:
datetime
- smc_monitoring.models.calendar.datetime_from_ms(ms)[source]
Convenience to return datetime from milliseconds
- Returns:
datetime from ms
- Return type:
datetime
- smc_monitoring.models.calendar.datetime_to_ms(dt)[source]
Convert an unaware datetime object to milliseconds. This datetime should be the time you would expect to see on the client side. The SMC will do the timestamp conversion based on the query timezone.
- Returns:
value representing the datetime in milliseconds
- Return type:
Monitors
The monitors package provides modules that represent individual monitoring
areas within the SMC monitoring API. Each monitor type extends
smc_monitoring.models.query.Query
to provide a consistent API for
adding filters and executing queries.
Blacklist
Blacklist Query provides the ability to view current blacklist entries in the SMC by target. Target is defined as the cluster or engine. Retrieved results will have a reference to the entry and hence be possible to remove the entry.
query = BlacklistQuery('sg_vm')
query.format.timezone('CST')
Optionally add an “InFilter” to restrict search to a specific field:
query.add_in_filter(
FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.2')])
An InFilter can also use a network based syntax:
query.add_in_filter(
FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.0/24')])
Or combine filters using “AndFilter” or “OrFilter”. Find an entry with source IP 2.2.2.2 OR 2.2.2.5:
ip1 = InFilter(FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.2')])
ip2 = InFilter(FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.5')])
query.add_or_filter([in_filter, or_filter])
Get the results of the query in the default TableFormat:
for entry in query.fetch_batch():
print(entry)
Delete any blacklist entries with a source IP within a network range of 3.3.3.0/24:
query = BlacklistQuery('sg_vm')
query.add_in_filter(
FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('3.3.3.0/24')])
for record in query.fetch_as_element(): # <-- must get as element to obtain delete() method
record.delete()
See also
smc_monitoring.models.filters
for more information on creating filters
- class smc_monitoring.monitors.blacklist.BlacklistEntry(**kw)[source]
Bases:
object
A blacklist entry represents an entry in the engines kernel table indicating that a source/destination/port/protocol mapping is currently being blocked by the engine. To remove a blacklist entry from an engine, retrieve all entries as element and remove the entry of interest by called
delete
on the element.The simplest way to use search filters with a blacklist entry is to examine the BlacklistQuery
field_ids
and use these constant fields as InFilter definitions on the query.- property blacklist_id
Blacklist entry ID. Useful if you want to locate the entry within the Management Client.
- Return type:
- delete()[source]
Delete the entry from the engine where the entry is applied.
- Raises:
DeleteElementFailed
- Returns:
None
- property dest_ports
Destination ports for this blacklist entry. If no ports are specified, ‘ANY’ is returned.
- Return type:
- property first_fetch
first fetch True means entry is part of initial data at first fetch
- Return type:
- property href
The href for this blacklist entry. This is the reference to the entry for deleting the entry.
- Return type:
- property protocol
Specified protocol for the blacklist entry. If none is specified, ‘ANY’ is returned.
- Return type:
- property source_ports
Source ports for this blacklist entry. If no ports are specified (i.e. ALL ports), ‘ANY’ is returned.
- Return type:
- class smc_monitoring.monitors.blacklist.BlacklistQuery(target, timezone=None, **kw)[source]
Bases:
Query
Query existing blacklist entries for a given cluster/engine. It is generally recommended to set your local timezone when making a query to convert the timestamp into a relevant format.
Note
Timezone can be in the following formats: ‘US/Eastern’, ‘PST’, ‘Europe/Helsinki’. More example time zone formats are available in the Logs view of the Management Client when you select Tools -> Time Zones.
Connections
A connection query returns all currently connected sessions on the given target.
Create a query to obtain all connections for a given engine:
query = ConnectionQuery('sg_vm')
Add a timezone to the query:
query.format.timezone('CST')
Add a filter to only get connections if the source address is 172.18.1.252:
query.add_in_filter(FieldValue(LogField.SRC), [IPValue('172.18.1.252')])
Only connections that match a specific service:
query.add_in_filter(FieldValue(LogField.SERVICE), [ServiceValue('TCP/443', 'UDP/53')])
Execute query and return raw results:
for records in query.fetch_raw():
...
Execute query and return as an Connection
element:
for records in query.fetch_as_element():
...
Retrieving live streaming results:
for records in query.fetch_live():
...
See also
smc_monitoring.models.filters
for more information on creating filters
- class smc_monitoring.monitors.connections.Connection(**data)[source]
Bases:
object
Connection represents a state table entry. This is the result of making a
ConnectionQuery
and usingfetch_as_element()
.- property engine
The engine/cluster for this state table entry
- Returns:
engine or cluster for this entry
- Return type:
- property first_fetch
first fetch True means entry is part of initial data at first fetch
- Return type:
- property state
State of the connection.
- Returns:
state, i.e. UDP established, TCP established, etc.
- Return type:
- class smc_monitoring.monitors.connections.ConnectionQuery(target, **kw)[source]
Bases:
Query
Show all current connections on the specified target.
- Variables:
field_ids (list) – field IDs are the default fields for this entry type and are constants found in
smc_monitoring.models.constants.LogField
- Parameters:
target (str) – name of target engine/cluster
Logs
LogQuery provides an interface to the SMC Log Viewer to retrieve data in real time or by batch.
There are a variety of settings you can configure on a query such as whether to execute a real time query versus a stored log fetch, time frame for the query, fetch size quantity, returned format style, specify which fields to return and adding filters to make a very specific query.
To make queries, first obtain a query object and optionally (recommended)
specify a maximum number of records to fetch (for non-real time fetches).
The default log query type is ‘stored’, and if a fetch_size
is not
provided, one batch of 200 records will be returned:
query = LogQuery(fetch_size=50)
If real time logs are preferred and set fetch_type='current'
(default is fetch ‘stored’ logs):
query = LogQuery(fetch_type='current')
You can also use the shortcut fetch_live
on the query:
query = LogQuery()
for result in query.fetch_live():
...
Note
If selecting fetch_size='current'
log queries will be
real-time and ignore the fetch_size
, time_range
, and
backwards
values if provided on the query.
You can also set a time_range on the query. There are convenience methods on a TimeFormat object to simplify adding a time range. When using time ranges, you should set the timezone on the query to the clients timezone:
query = LogQuery(fetch_size=50)
query.time_range.last_five_minutes()
query.format.timezone('CST')
You can also use custom time ranges to search between a specific period of time.
This is done by providing a smc_monitoring.models.calendar.TimeFormat
instance to the Query constructor, or by modifying the query time_range
attribute.
The TimeFormat object takes a ‘naive’ datetime object for start and end times.
The start and end times must also be in milliseconds.
Example of finding all records on 9/2/2017 from 06:25:30 to 06:26:30 in the local time zone CST:
dt_start = datetime(2017, 9, 2, 6, 25, 30, 0)
dt_end = datetime(2017, 9, 2, 6, 26, 30, 0)
query = LogQuery()
query.format.timezone('CST') # <--- Set the timezone on the query!
query.time_range.custom_range(
datetime_to_ms(dt_start),
datetime_to_ms(dt_end))
See also
smc_monitoring.models.calendar.TimeFormat
for more examples
and information on using a TimeFormat in a query.
Adding filters to a query can be achieved by using add_XX_filter convenience
methods or by calling update_filter
with the filter object.
For example, customizing the fields returned using query.format.field_ids
, and
filtering for only HIGH alerts with a source address of 192.168.4.84:
query = LogQuery(fetch_size=10)
query.format.timezone('CST')
query.format.field_ids([LogField.TIMESTAMP, LogField.ACTION, LogField.SRC, LogField.DST])
query.add_and_filter(
[InFilter(FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH)]),
InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')])])
See also
smc.monitoring.filters
for information on how to use and
combine filters for a query.
- class smc_monitoring.monitors.logs.LogQuery(fetch_type='stored', fetch_size=None, backwards=True, format=None, time_range=None, **kw)[source]
Bases:
Query
Make a Log Query to the SMC to fetch stored log data or monitor logs in real time.
- Variables:
field_ids (list) – field IDs are the default fields for this entry type and are constants found in
smc_monitoring.models.constants.LogField
- Parameters:
fetch_type (str) – ‘stored’ or ‘current’
fetch_size (int) – max number of logs to fetch
backwards (bool) – by default records are returned from newest to oldest (backwards=True). To return in opposite direction, set backwards=False. Default: True
format (format type from
smc_monitoring.models.formats
(default: TextFormat)) – A format object specifying format of return datatime_range (TimeFormat) – time filter to add to query
servers (list[str,Element]) – A list of href or server elements for which to query
- fetch_batch(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>)[source]
Fetch a batch of logs and return using the specified formatter. Formatter is class type defined in
smc_monitoring.models.formatters
. This fetch type will be a single shot fetch (this method forcesfetch_type='stored'
). Iffetch_size
is not already set on the query, the default fetch_size will be 200.- Parameters:
formatter – Formatter type for data representation. Any type in
smc_monitoring.models.formatters
.- Returns:
generator returning data in specified format
- fetch_live(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>)[source]
View logs in real-time. If previous filters were already set on this query, they will be preserved on the original instance (this method forces
fetch_type='current'
).- Parameters:
formatter – Formatter type for data representation. Any type in
smc_monitoring.models.formatters
.- Returns:
generator of formatted results
- fetch_raw()[source]
Execute the query and return by batches. Optional keyword arguments are passed to Query.execute(). Whether this is real-time or stored logs is dependent on the value of
fetch_type
.- Returns:
generator of dict results
- property fetch_size
Return the fetch size for this query. If fetch size is set to 0, the query will be aborted after the first response message. If the fetch_size is None, it is considered undefined which indicates there is no fetch bound set on this query (i.e. fetch all).
- ..note:: It is recommended to provide a fetch_size to limit the
results when doing a ‘stored’ query.
- Returns:
configured fetch size for this query
- Return type:
Routes
Query the current routing table entries.
Create a query to obtain all connections for a given engine:
query = RoutingQuery('sg_vm')
Add a timezone to the query:
query.format.timezone('CST')
Add a filter to only routes for destination network 192.168.4.0/24:
query.add_in_filter(FieldValue(LogField.ROUTENETWORK), [IPValue('192.168.4.0')])
Only routes that use a specific gateway:
query.add_in_filter(FieldValue(LogField.ROUTEGATEWAY), [IPValue('172.18.1.200')])
Execute query and return raw results:
for records in query.fetch_batch():
...
Execute query and return as an RoutingView
element:
for records in query.fetch_as_element():
...
See also
smc_monitoring.models.filters
for more information on creating filters
- class smc_monitoring.monitors.routes.RoutingQuery(target, **kw)[source]
Bases:
Query
Show all current dynamic and static routes on the specified target.
- Variables:
field_ids (list) – field IDs are the default fields for this entry type and are constants found in
smc_monitoring.models.constants.LogField
- Parameters:
target (str) – name of target engine/cluster
- class smc_monitoring.monitors.routes.RoutingView(**data)[source]
Bases:
object
A Routing View represents an entry in the current routing table. This is the result of making a
RoutingQuery
and usingfetch_as_element()
.- property first_fetch
first fetch True means entry is part of initial data at first fetch
- Return type:
- property timestamp
Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:
query.format.timezone('CST')
:return timestamp in string format :rtype: str
SSLVPN
SSLVPN currently connected users.
Create a query to obtain all connections for a given engine:
query = SSLVPNQuery('sg_vm')
Add a timezone to the query:
query.format.timezone('CST')
Execute query and return raw results:
for records in query.fetch_batch():
...
Execute query and return as an SSLVPNUser
element:
for records in query.fetch_as_element():
...
See also
smc_monitoring.models.filters
for more information on creating filters
- class smc_monitoring.monitors.sslvpn.SSLVPNQuery(target, **kw)[source]
Bases:
Query
Show all current SSL VPN connections on the specified target.
- Variables:
field_ids (list) – field IDs are the default fields for this entry type and are constants found in
smc_monitoring.models.constants.LogField
- Parameters:
target (str) – name of target engine/cluster
- class smc_monitoring.monitors.sslvpn.SSLVPNUser(**data)[source]
Bases:
object
Connection represents a state table entry. This is the result of making a
SSLVPNQuery
and usingfetch_as_element()
.- property engine
The engine/cluster for this state table entry
- Returns:
engine or cluster for this entry
- Return type:
- property first_fetch
first fetch True means entry is part of initial data at first fetch
- Return type:
- property session_expiration
Time the session expires. It is recommended that you add a timezone to the query to present this in human readable format:
query.format.timezone('CST')
- Return type:
- property session_start
Time the session started. It is recommended that you add a timezone to the query to present this in human readable format:
query.format.timezone('CST')
- Return type:
Users
Get active users on target cluster/engine.
Create a query to obtain all users for a given engine:
query = UserQuery('sg_vm')
Add a timezone to the query:
query.format.timezone('CST')
Execute query and return raw results:
for records in query.fetch_batch():
...
Execute query and return as a User
element:
for records in query.fetch_as_element():
...
See also
smc_monitoring.models.filters
for more information on creating filters
- class smc_monitoring.monitors.users.User(**data)[source]
Bases:
object
User mapping currently in user cache on specified target. This is the result of making a
UserQuery
and usingfetch_as_element()
.- property domain
SMC Domain that this user record belongs to
- Returns:
name of SMC domain, ‘Shared’ is default
- Return type:
- property engine
The engine/cluster for this state table entry
- Returns:
engine or cluster for this entry
- Return type:
- property expiration
Expiration time for this user entry. It is recommended to add a timezone to the query to display this field in the client local time.
- Returns:
expiration time for this user authentication entry
- Return type:
- property first_fetch
first fetch True means entry is part of initial data at first fetch
- Return type:
- property timestamp
Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:
query.format.timezone('CST')
:return timestamp in string format :rtype: str
- class smc_monitoring.monitors.users.UserQuery(target, **kw)[source]
Bases:
Query
Show all authenticated users on the specified target.
- Variables:
field_ids (list) – field IDs are the default fields for this entry type and are constants found in
smc_monitoring.models.constants.LogField
- Parameters:
target (str) – name of target engine/cluster
VPNs
Get all active VPN SA’s.
Create a query to obtain all connections for a given engine:
query = VPNSAQuery('sg_vm')
Add a timezone to the query:
query.format.timezone('CST')
Execute query and return raw results:
for records in query.fetch_batch():
...
Execute query and return as a VPNSecurityAssoc
element:
for records in query.fetch_as_element():
...
Delete a VPN SA:
query = VPNSAQuery('sg_vm')
for sa in query.fetch_as_element():
sa.delete()
See also
smc_monitoring.models.filters
for more information on creating filters
- class smc_monitoring.monitors.vpns.VPNSAQuery(target, **kw)[source]
Bases:
Query
Show all current VPN SA’s on the specified target.
- Variables:
field_ids (list) – field IDs are the default fields for this entry type and are constants found in
smc_monitoring.models.constants.LogField
- Parameters:
target (str) – name of target engine/cluster
- class smc_monitoring.monitors.vpns.VPNSecurityAssoc(**data)[source]
Bases:
object
A VPN Security Association represents a currently connected VPN endpoint. This is the result of making a
VPNSAQuery
and usingfetch_as_element()
.- property first_fetch
first fetch True means entry is part of initial data at first fetch
- Return type:
- property negotiation_role
Role for this tunnel entry.
- Returns:
Negotiation role, i.e. Initiator, Responder, etc.
- Return type:
- property protocol
WHich protocol is associated with this tunnel entry.
- Returns:
IP protocol for tunnel, i.e. ESP/UDP
- Return type:
- property sa_type
SA Type for this VPN tunnel. Each VPN tunnel will typically have at least two entries, one for IPSEC and another for IKE.
- Return type:
Alerts
ActiveAlert Query provides the ability to view current alert entries from the alert log viewer. When creating the query, you must specify a target which speifies the SMC domain for which to retrieve the alerts.
A basic alert query using a local timezone example:
query = ActiveAlertQuery('Shared Domain')
query.format.timezone('CST')
You can also use standard filters to specify a more exact match, for example, showing alerts with a severity of CRITICAL:
query.add_in_filter(
FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.CRITICAL)])
- class smc_monitoring.monitors.alerts.ActiveAlertQuery(target='Shared Domain', timezone=None)[source]
Bases:
Query
Active Alert Query is an interface to the alert log viewer in Log Server. This query type provides the ability to fetch and filter on active alerts.
You can create a new query specifying a valid timezone abbreviation:
query = ActiveAlertQuery('Shared Domain', timezone='CST')
Or alternatively no timezone:
query = ActiveAlertQuery('DomainFoo')
- Parameters:
- class smc_monitoring.monitors.alerts.Alert(**data)[source]
Bases:
object
Alert definition returned from specified domain. This is the result of making a
ActiveAlertQuery
and usingfetch_as_element()
.- property engine
The engine/cluster for this state table entry
- Returns:
engine or cluster for this entry
- Return type:
- property timestamp
Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:
query.format.timezone('CST')
:return timestamp in string format :rtype: str