Tracking and Reporting

Introduction

We need to report to a number of different stakeholder groups to demonstrate the value we provide to the Australian climate research community.

Reporting necessitates tracking what is used and by which user group.

Tracking usage helps to identify which tools are being used by the community and how they use them, to focus our efforts, and also highlight where training and outreach may be required to improve uptake.

Tracking

Why?

The core services and infrastructure we use at ACCESS-NRI (NCI, Github, Zenodo) don’t provide the tracking tools we need to demonstrate the value we provide (as described above) e.g. NCI can’t provide some data, or doesn’t provide data in a way we can utilise efficiently. GitHub provides some data, but doesn’t save it, the same applies for Zenodo.

What we track

Forum stats: number of users, number of posts, help stuff?
NCI resource usage
GitHub stats (in progress)
Zenodo stats (in progress)
Data access via intake catalogues
Datastore creation via intake utilities
Model runs via payu (in progress)

How?

Scheduled running of reporting tools, parse output and upload to database: NCI, GitHub, Zenodo
Run queries against internal forum database to generate statistics
Embed telemetry in ACCESS-NRI tools and report usage, e.g. data accessed via catalogues, what models ran, experiment ID, performance and resources consumed.
We do not collect NCI logins: all telemetry is completely anonymous.

Reporting

Who do we report to and why?

Government and partners: to justify their investment.
ACCESS-NRI Board: who have oversight to make sure we’re doing what we promised.
Community: to demonstrate our value and encourage the use and uptake of what we provide.
Internally: manage resources, both internal and storage/compute provided to community

What do we report

Externally: only aggregated data by institutional affiliation, or broad groupings such as academic/private/overseas. Examples: number of posts on forum, number of accesses of ACCESS-NRI supported data, number of times ACCESS-NRI released models are run
Internally: we will generate more granular data insights using Grafana dashboards to allow efficient administration of shared resources.

What do we not report?

No individual level statistics are collected, nor will be generated or published.
No activities at an individual level without consent, e.g. case-studies

Data security?

We only collect the minimum information required, we do not store personal names or contact information
Data will be stored in secured databases to industry best standard using the NRI Nectar Cloud Service
Un-aggregated data will only be used internally by a small number of ACCESS-NRI staff for the purposes of system maintenance, report production, etc. We only transmit aggregated data to external services, never unaggregated data.
All reports and services will be stored and transmitted securely using encrypted protocols (HTTPS, SSH, etc.)