Getting Rule utilization data in nested hierarchy of device groups

[wrgeorge1983]

I've banged my head against this for a while now, and simply not been able to figure out how to rule hitcount data.

Given a device group hierarchy as shown:

shared
  - Branches
    - Site_A 
    - Site_B 
    - ...

with 2 firewalls (HA pair) assigned to Site A, and a different set assigned to Site B, etc

with rules defined like this:

- Rules 1-100 defined in Branches
- Rules 101-155 defined in Site_A
- Rules 156-185 defined in Site_B
- etc (not strictly consistent, some sites have zero rules, some have 100)

I want to collect information about rule hitcount, appID statistics, etc for Rules both at the global level, as well as for individual sites. The goal is to report on and answer questions such as:

Rule 4 is unused at any site
Rule 10 is used heavily (with actual numbers) with detected AppIDs x, y, z across all sites, accounting for 7% of all traffic
Rule 8 accounts for 40% of traffic at SiteA, but only 5% of traffic at SiteB
Rule 160 (which is only present at SiteB) accounts for all the other 88% of traffic not covered by rules 10 and 8, and these are the AppIDs detected....

stuff like that.

All this information IS available in Panorama via GUI, but only with lots of repetitive clicking and manual data collection so I'm seeking to automate it. So far I have been able to walk the Device Group hierarchy and show defined rules at each level, but haven't been able to get hitcount or AppID information.

I know the HitCount and RulebaseHitCount objects exist, but can't figure out how to get any actual data in them.

For example this is the real usage data for a specific rule viewed in Panorama in the Branches Device Group:

doing this to pick out that same rule and (I think) populate the hitcount information:

report = Report(hostname, username, api_key)
p = report.p  # Panorama Object
p.refresh_devices(add=True)
dg = p.find("Branches")  # DeviceGroup Object
prb = dg.add(PreRulebase())
srs = SecurityRule.refreshall(prb)  # List(SecurityRule)
guest_wifi_rule = srs[-1]
rbc = RulebaseHitCount(dg)
rbc.refresh('security')

gets me the hitcount object that looks like this:

inspect(guest_wifi_rule.opstate.hit_count)

╭─────────────────────────────────── <class 'panos.policies.HitCount'> ───────────────────────────────────╮
│ Hit count operational data.                                                                             │
│                                                                                                         │
│ ╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ │ <panos.policies.HitCount object at 0x000001F6E52524D0>                                              │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                         │
│                      FIELDS = (                                                                         │
│                                   ('latest', 'latest', 'str'),                                          │
│                                   ('hit_count', 'hit-count', 'int'),                                    │
│                                   ('last_hit_timestamp', 'last-hit-timestamp', 'int'),                  │
│                                   ('last_reset_timestamp', 'last-reset-timestamp', 'int'),              │
│                                   ('first_hit_timestamp', 'first-hit-timestamp', 'int'),                │
│                                   ('rule_creation_timestamp', 'rule-creation-timestamp', 'int'),        │
│                                   (                                                                     │
│                                       'rule_modification_timestamp',                                    │
│                                       'rule-modification-timestamp',                                    │
│                                       'int'                                                             │
│                                   )                                                                     │
│                               )                                                                         │
│         first_hit_timestamp = None                                                                      │
│                   hit_count = None                                                                      │
│          last_hit_timestamp = None                                                                      │
│        last_reset_timestamp = None                                                                      │
│                      latest = None                                                                      │
│                        name = 'None'                                                                    │
│                         obj = <SecurityRule GUEST_WIFI_Access_Allow 0x1f6e5252530>                      │
│     rule_creation_timestamp = None                                                                      │
│ rule_modification_timestamp = None                                                                      │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯

and the security rule itself looks like:

inspect(guest_wifi_rule)

╭─────────────────────────────── <class 'panos.policies.SecurityRule'> ────────────────────────────────╮
│ Security Rule                                                                                        │
│                                                                                                      │
│ ╭──────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ │ <SecurityRule GUEST_WIFI_Access_Allow 0x1f6e5252530>                                             │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                      │
│                             action = 'allow'                                                         │
│                        application = ['any']                                                         │
│                           category = ['any']                                                         │
│                       CHILDMETHODS = ()                                                              │
│                           children = []                                                              │
│                         CHILDTYPES = ()                                                              │
│                     data_filtering = None                                                            │
│                        description = None                                                            │
│                        destination = ['any']                                                         │
│                destination_devices = ['any']                                                         │
│ disable_server_response_inspection = None                                                            │
│                           disabled = None                                                            │
│                      file_blocking = None                                                            │
│                           fromzone = ['GUEST']                                                       │
│                              group = None                                                            │
│                          group_tag = None                                                            │
│                            HA_SYNC = True                                                            │
│                       hip_profiles = None                                                            │
│                    HIT_COUNT_STYLE = 'security'                                                      │
│                   icmp_unreachable = None                                                            │
│                            log_end = None                                                            │
│                        log_setting = 'AWS-PANO'                                                      │
│                          log_start = None                                                            │
│                               NAME = 'name'                                                          │
│                               name = 'GUEST_WIFI_Access_Allow'                                       │
│                 negate_destination = None                                                            │
│                      negate_source = None                                                            │
│                      negate_target = False                                                           │
│                            opstate = <panos.base.OpStateContainer object at 0x000001F6E5251690>      │
│                           OPSTATES = {                                                               │
│                                          'audit_comment': <class 'panos.policies.RuleAuditComment'>, │
│                                          'hit_count': <class 'panos.policies.HitCount'>              │
│                                      }                                                               │
│                             parent = <PreRulebase 0x1f6e4ed0b80>                                     │
│                               ROOT = 1                                                               │
│                           schedule = None                                                            │
│                            service = ['any']                                                         │
│                             source = ['any']                                                         │
│                     source_devices = ['any']                                                         │
│                        source_user = ['any']                                                         │
│                            spyware = None                                                            │
│                             SUFFIX = "/entry[@name='%s']"                                            │
│                                tag = ['GUEST_WIFI']                                                  │
│                             target = None                                                            │
│                    TEMPLATE_NATIVE = False                                                           │
│                             tozone = ['any']                                                         │
│                               type = None                                                            │
│                                uid = 'GUEST_WIFI_Access_Allow'                                       │
│                      url_filtering = None                                                            │
│                               uuid = '4d440c03-3a75-4252-9a4a-f9be29ed5a48'                          │
│                              virus = None                                                            │
│                               vsys = 'Branches'                                                      │
│                      vulnerability = None                                                            │
│                  wildfire_analysis = None                                                            │
│                              XPATH = '/security/rules'                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────╯

So far I've tried many different variations of this, but haven't been able to pull any actual hitcount values. Any idea what I'm doing wrong?

[BatD2]

It will be helpful if someone form the team answers this. I am also having the same query

[jvictorgonc]

HELLO,
i dont know if my answer would be useful, but i can see hit counts from my firewall rules using the following script:

from panos.policies import SecurityRule, Rulebase, NatRule,RulebaseHitCount

rulebase = Rulebase()
fw_device = Firewall(hostname='labfw1', 
                     api_username='lab-admin', 
                     api_password='lablab123')
fw_device.add(rulebase)
rules = SecurityRule.refreshall(rulebase)
hitcount = RulebaseHitCount(rulebase)
hitcountrules = hitcount.refresh('security') #<-- the result is a big dictionary with panos.policies.HitCount objects like this: "<panos.policies.HitCount object at 0x000001BB50A9B990>" 
for k,v in hitcountrules.items(): # So, if you iter for dictionary, you will see that the object has some parameters as in the figure below
    print(f'RULENAME: {k} | HITCOUNT: {v.hit_count}')