module libpyRR2_datafiles

Parent Previous Next


Module libpyRR2_datafiles

The module libpyRR2 is a standalone/external module.

You can load/import it into any Python environment, e.g. Houdini or rrPythonconsole.


Examples

Please see folder [RR]\SDK\External\Python\ for some examples.


Automated collection?

There is an example how to create weekly stats with a job in RR.

Please start the rrSubmitter, menu Scripted Jobs, execute "Create Weekly Stats" and submit the job.

It uses this module to create a .csv (e.g. Excel) table each week with the latest stats.




Module description



Module libpyRR2 functions:

Type

Name

Description

char * 

getRRModuleVersion()

Requires to call any jobGet..() function.

string

getRRFolder_cfgGlobal()

Returns the path [RR]/sub/cfg_global.  
It has to be executed from a machine in an RR environment (System variable RR_ROOT is defined). Otherwise it returns an empty string.

string

getRRFolder_cfgUser()

Returns the path [RR]/sub/cfg_user.  

It has to be executed from a machine in an RR environment (System variable RR_ROOT is defined). Otherwise it returns an empty string.

string

getRRFolder_stats()

Returns the path [RR]/sub/stats. 

 It has to be executed from a machine in an RR environment (System variable RR_ROOT is defined). Otherwise it returns an empty string.

string

getRRFolder_bin()

Returns the path [RR]/bin/[OS]

 It has to be executed from a machine in an RR environment (System variable RR_ROOT is defined). Otherwise it returns an empty string.



Module libpyRR2 classes :

Class

Description

_dailyStats_data

Job information for each project, user and render application that was used.
Informations like

  • Memory usage
  • CPU usage. 
  • Number of jobs rendering, idle, finished, ..
  • Number of frames rendered and frames to be rendered.

_rrGlobalStats

Same as you see in rrControl tab "Global Info".

Clients enabled, disabled, rendering.

Jobs rendering, waiting

Frames rendered, waiting

Server CPU usage

_rrClientStats_data

rrClient stats with 

  • CPU usage
  • memory usage
  • current job
  • ...

_ClientGroupList

The client group list.

_frameStatJob

Same data as the the "frames rendered" table in rrControl.
Per frame information includes 

  • Render time
  • Memory  (5 times per frame)
  • CPU usage (5 times per frame)

_StatsRenderTime

Every time a job is finished, it adds its average frame time to a list, separated by each layer of each project.
It includes the last 100 finished jobs of each layer name of each project.
(For example the the last 100 jobs of render layer "Ambient Occlusion" of project "FancyDrink-Commercial")


_AppLicenses_v8

This class contains the data of the rrConfig tab "Render Lic".












Class _dailyStats_data

This class loads the statistic files from the folder [RR]\sub\stats\server\

There is one file per day.

Each file contains stats about each user, project and render application.

The stats are collected every 15 minutes during the day.



Class _dailyStats_data functions:

Return type

Name

Description

list<_jobStatDay>

getJobsByType(id)

Returns a list of statistics.

The id value can be:

0: User stats

1: Project stats

2: Render app stats

int

jobTypeCount()

Returns the number of different stat lists. Which is 3.

bool

loadFromFile(string filename, bool failifnofile)

Loads a stat file. failIfNoFile should be True.

string

getError()

Returns the error message in case the loadFromFile() fails.




Class _jobStatDay


Class _jobStatDay parameter:

Return type

Name

Description

rrString

name

Name of the user, project or render app

byte

type

This value can be

0: User stats

1: Project stats

2: Render app stats



Class _jobStatDay functions:

Return type

Name

Description

_jobStat

getTimeSlot(int idx)

Get the statistics data at array position idx.

Each array item contains the stats for 15 minutes of that day:

0: 00:00- 00:15

1: 00:15- 00:30

2: 00:30- 00:45

...

int

timeSlotCount()

Returns the maximum number of possible _singleStat values. 

It always returns 96 for this RR version.   (24h/96 = 15 min)



Class _jobStat


Class _jobStat parameter:

Return type

Name

Description

bool

slotSet

True if this time slot contains any valid data

float

avPSUsage

Average PS usage

float

maxPSUsage

Maximum PS usage

float

PSneeded

An estimate how many PS would be needed to render all unassigned segments

float

renderLicNeeded

An estimate how many render licenses would be needed to render all unassigned segments

UInt16

jobsWaiting

Number of jobs in wait state

UInt16

jobsDisabled

Number of jobs in disabled state

UInt16

jobsIdle

Number of jobs in idle state

UInt16

jobsRendering

Number of jobs in rendering state

UInt16

jobsDone

Number of jobs in done state

UInt16

timesSended

Number of times a job was sended

UInt16

timesFinished

Number of times a job finished

UInt32

avMemUsageMB

Average memory usage

UInt32

maxMemUsageMB

Maximum memory usgae

UInt32

avFrameTime

Average frame time

UInt32

maxFrameTime

Maximum frame time

UInt32

framesRendered

Frames rendered during these 15 minutes

UInt32

framesLeft

Frames left to render




Class _rrGlobalStatsLS

This class loads the file [RR]\sub\stats\server_stats.db

This statistics file is used to create the server stats in rrControl, tab Global Info.

There are three different stats in this file.




Class _rrGlobalStatsLS parameter:

Return type

Name

Description

int64

lastSaveDay

The time the last index of the day array was saved. This value is the system time in the current time zone divided by 90.

Use currentLocalTime() to get the current local time.

int64

lastSaveMonth

The time the last index of the day array was saved. This value is the system time in the current time zone divided by 1800 (=30 minutes).

Use currentLocalTime() to get the current local time.

int64

lastSaveYear

The time the last index of the day array was saved. This value is the system time in the current time zone divided by 21600 (=6 hours).

Use currentLocalTime() to get the current local time.

int64

currentLocalTime()

Returns the current system time in the local time zone in seconds. 
The system time is based on the default unix timestamp (seconds since 00:00, Jan 1, 1970. The "Epoch"). 



Class _rrGlobalStatsLS functions:

Return type

Name

Description

_rrGlobalStats_item

day(int idx)

Get the day stats at index idx.

Index 0 returns the first 60s of the day, the time from 00:00.01 to 00:01.00.


The stats are constantly overwritten with the information right now, therefore an index larger than the current time returns stats of yesterday.

Example: 
It is 18:00 right now. If you get the stats from 17:00, then you have the stats from 1 hour ago. If you get the stats from 19:00, then you get the stats from 23h ago.

Note: The rrServer has to be running to continue to save stats. Otherwise check lastSaveDay.

int

dayMax()

Max number of stat values in the array. It returns 1440 for the current version.

_rrGlobalStats_item

month(int idx)

The server adds the stats of the last 30 minutes to the last index (959) of the array and pushes all other stat items down in the array.

Therefore an index of monthMax()-1=959 returns the stats of the last completed 30 minutes.

e.g. if the current time is 18:15, then the last stats are from 17:30 - 18:00

Note: The rrServer has to be running to continue to save stats. Otherwise check lastSaveMonth.

int

monthMax()

Max number of stat values in the array. It returns 1440 for the current version.

_rrGlobalStats_item

year(int idx)

The server adds the stats of the last 6 hours to the last index (1439) of the array and pushes all other stat items down in the array.

Therefore an index of yearMax()-1=1439 returns the stats of the last completed quater day minutes.

e.g. if the current time is 17:15, then the last stats are from 06:00 - 12:00

Note: The rrServer has to be running to continue to save stats.Otherwise check lastSaveYear.

int

yearMax()

Max number of stat values in the array. It returns 1440  for the current version.

bool

loadFromFile(string FileName)

Loads the stats from file FileName

string

getError()

Returns the error message in case the loadFromFile() fails.



Class _rrGlobalStats_item


Class _rrGlobalStats_item parameter:

Return type

Name

Description

bool

slot_used

Is this data valid? If not, then this data is crap or from any other day/time.

float

clients_needed

A estimate how many clients would be needed to start all unassigned frame segments.

float

framesPerMin

Number of frames rendered during this time segment  (90seconds, 30 minutes or 6hours depending on the stats type)

UInt32

framesPerMin_3D

Same as framesPer90s, but 3D jobs only

UInt32

framesPerMin_Comp

Same as framesPer90s, but Comp jobs only

UInt32

framesPerMin_Archive

Same as framesPer90s, but Archive jobs only

UInt16

clients_free

Number of free clients

UInt16

clients_rendering

Number of clients rendering

UInt16

clients_disabled

Number of disabled clients

UInt16

CpuProcess

CPU usage of the rrServer app  (0-100%)

UInt16

CpuMachine

CPU usage of the machine (0-100%)

UInt16

serverTemperature

Temperature of the first HDD in the rrServer machine

UInt16

clients_avTemperature

Average temperature of the first HDD of all rrClients

UInt32

jobs_unfinished

Number of nfinished jobs

UInt32

jobs_rendering

Number of jobs rendering

UInt32

jobs_total

Total number of jobs

UInt32

serverMemUsage_Machine

Memory usage of the rrServer machine.

UInt32

serverMemUsage_Process

Memory usage of the rrServer application.

UInt32

serverMemUsage_TCPSocket

Memory usage of the 1st rrServers TCP socket buffer

UInt32

serverMemUsage_TCPData

Memory usage of the 2nd rrServers TCP socket buffer

UInt32

serverMemUsage_JobBuffer

Memory usage for all job data

UInt32

serverMemUsage_TCPDataCompressed

Max memory for a TCP socket buffer after compression

UInt32

frameTime

Average frame time of all jobs

UInt32

framesLeft

Summed frames to be rendered from all jobs


Class _rrClientStats_data

The client saves statistics per day.



Class _rrClientStats_data parameter:

Return type

Name

Description

UInt32

StructureID

Should be the hex value A101



Class _rrClientStats_data functions:

Return type

Name

Description

_rrClientStats_day

days(int idx)

Get the data per day. 
Index  (daysMax() - 1) is the last day written into the stats file. If the client is currently running, then this is today.

Index  (daysMax() - 2) is the day before.
Check days(...).daySet and days(...).day to verify that you got some data and that it is the right day.

int

daysMax()

The maximum number of days in this array (180).

bool

loadFromFile(string FileName)

Loads the stats from file FileName

string

getError()

Returns the error message in case the loadFromFile() fails.






Class _rrClientStats_day


Each day contains two different sets of data, one is collected every 20 seconds, the other one collected every hour.



Class _rrClientStats_day parameter:

Return type

Name

Description

bool

daySet

Contains this day any useful data?

(e.g. if the client was not online that day, daySet is false)

UInt32

day

The current day in the default unix timestamp (days since Jan 1, 1970).  
Note: Most time functions return the time in seconds since 1970, so divide that value with /60/60/24.


Class _rrClientStats_day functions:

Return type

Name

Description

_rrClientStats_day_data

data(int idx)

This data is collected every 30 seconds.
Index 0 are the first 30 seconds of the day, Index 1 the 30-60 seconds of the day, ...

int

dataMax()

The maximum number of values in the data array. (2880 for the current version, one data item every 30 seconds)

_rrClientStats_day_data_hourly

dataHourly(int idx)

Index 0 is the first hour of the day ( 00:00  - 01:00 )

int

dataHourlyMax()

The maximum number of values in the hourly array (24)




Class _rrClientStats_day_data_hourly


The hourly information is collected for each job thread that the client has enabled.


Class _rrClientStats_day_data_hourly functions:

Return type

Name

Description

_rrClientStats_day_data_hourly_jobthread

jobThread(int idx)

Get the data for each job thread.

int

jobThreadCount()

The number of job threads the client was running in this hour.


Class _rrClientStats_day_data_hourly_jobthread


Class _rrClientStats_day_data_hourly_jobthread parameter:

Return type

Name

Description

UInt16

framesDone

Frames done in this hour

UInt8

jobCrashed

Number of times a job crashes

UInt8

jobAborted

Number of times a job was aborted

UInt8

jobSuccessful

Number of times a job rendered successfully



Class _rrClientStats_day_data_20s


Class _rrClientStats_day_data parameter:

Return type

Name

Description

bool

userLoggedIn

Was a user logged in? 

rrPercent

cpuTotal

CPU usage as percentage of the system.

The value is exported as float  (1.0 = 100%)

rrPercent

gpuTotal

GPU usage as percentage of the system. 

The value is exported as float  (1.0 = 100%)

UInt16

memTotal10MB

Total memory usage in MiB, divided by 10.

Int16

networkInMB

Total MiB of data received from the network.  Not supported yet, returns -1.

Int16

networkOutMB

Total MiB of data send into the network.  Not supported yet, returns -1.

UInt8

networkInSpeedMaxMB

Maximum MiB/s.   Currently not supported yet, returns 0.



Class _rrClientStats_day_data functions:

Return type

Name

Description

_rrClientStats_day_data_jobthread

jobThread(int idx)

Returns the data of a job thread

int

jobThreadCount()

The number of job threads the client was running in this time slot.

bool

hasData()

Returns if this time slot was set. 



Class _rrClientStats_day_data_jobthread


Class _rrClientStats_day_data_jobthread parameter:

Return type

Name

Description

byte

status

Job status. See enumeration _Status

rrPercent

cpuRender

CPU usage of this job.

The value is exported as float  (1.0 = 100%)

UInt16

memRender10MB

Memory usage of the render job in MB, divided by 10

UInt32

jobIDShort

The ID of the job in a shortened representation (The internal job ID in RR is usually a 64bit number. In this case truncated to 32bit only)



Class _rrClientStats_day_data_jobthread functions:

Return type

Name

Description

string

jobIDShortStr() 

The jobIDShort as human readable string {ID} 




Class _frameStatJob


Class _frameStatJob functions:

Return type

Name

Description

bool

loadFromFile(string FileName)

Loads the stats from file FileName

string

getError()

Returns the error message in case the loadFromFile() fails.

_frameStat

frame(int frame)

Get the stats of frame X. The first frame has index number 0. It is not the frame number in the scene file.

int

frameCount()

Number of frames for this job.






Class _frameStat


Per frame data


Class _frameStat parameter:

Return type

Name

Description

_rrTime32

logFileStart

The time when the render logfile was started.

It is the same time as in the filename.

int

logFileStartFrame

The first frame of the render log.

It is the same frame number as in the filename.

UInt16

renderTime5Sec

The render time of the frame in seconds, divided by 5.

UInt16

frameNrJob

The frame number of the job.

rrString8_50

client

Name of the client

Uint8

averageValue

Sometimes the client is not able to get exact values per frames.

In this case it collects the data of the whole frame segment and averages the values.

The frame segment contains of averageValue frames.



Class _frameStat functions:

Return type

Name

Description

_frameStat_part

part()

Stats are collected multiple times while a frame is rendered.

If a frame takes for example 10 minutes to render, then each of the 5 part represents 2 minute of the render.

int

partMax()

Max number of parts for each frame (5)




Class _frameStat_part


Class _frameStat_part parameter:

Return type

Name

Description

UInt16

jobMemoryMB

Memory usage of job

rrPercent

jobCPU

CPU usage of the job

rrPercent

nonJobCPU

Other CPU usage of the machine

UInt8

renderThreads

Number of job threads rendering on the client at this moment


Class _StatsRenderTime


Class _StatsRenderTime parameter:

Return type

Name

Description

rrTime

settingsFileDate

Date/Time of the group settings file loaded/saved.

UInt16

count

Get/set the number of groups



Class _StatsRenderTime functions:

Return type

Name

Description

_StatsRenderTime_project

getProject(int idx)

Get project at idx.

bool

loadFile(string FileName)

Loads the stats from file FileName.

bool

saveFile(string FileName)

Saves the stats into file FileName.

string

getError()

Returns the error message in case the loadFile() or saveFile() fails.



Class _StatsRenderTime_project


Class _StatsRenderTime_project parameter:

Return type

Name

Description

rrTime

lastUsed

The last time a job was added to this list.

string50

name

Name of the project



Class _StatsRenderTime_project functions:

Return type

Name

Description

_StatsRenderTime_layer

getLayer(int idx)

Get layer at idx.

int

layerCount()

Returns the number of layers in this list.



Class _StatsRenderTime_layer


Class _StatsRenderTime_layer  parameter:

Return type

Name

Description

int16

jobsAdded

Number of jobs that have been added. This is NOT the number of jobs in this list.

string50

name

Name of the layer



Class _StatsRenderTime_layer functions:

Return type

Name

Description

UInt16

getFrameTime(int idx)

Get the frame time at idx. The frame time is in PS*s.

int

timesCount()

Returns the number of frame times in this list. The latest job is always at position 0.




Class _server_clientInfo


Class _server_clientInfo functions:

Return type

Name

Description

_Client_Info_save

info(int idx)

Get the _Client_Info_save at the index. Note: The index does not have to match the client order in the current client list in RR.

bool

loadFromFile(string FileName)

Loads the infromation from file filename. The current filename is RR/sub/stats/clientIPstats82.db

bool

getError()

Returns the error message in case the loadFromFile() fails..

int

count()

The number of entries stores in this array.



Class _Client_Info_save


Class _Client_Info_save parameter:

Return type

Name

Description

rrTime

lastConnect

The last time the rrClient reported a status.

rrTime

lastWOLTry

Last time a WOL command was send to the machine.

uint8

WOLTryCount

Number of times the WOL command was executed without success.

Note: The last WOL command could still be in progress and is therefore not successful yet.

rrString50

saveNameCopy

Name of the rrClient

rrString10

saveVersionCopy

Last known version of the rrClient.



Class _Client_Info_save functions:

Return type

Name

Description

rrIP

lastIPs(int idx)

Get the IPs the rrClient had.
Index 0 is the latest IP. The IP is stored if the rrClient had a different IP during its latest status report.

rrTime

lastIPDate(int idx)

Get the time the rrClient reported this new IP.

_Client_WOLInfo

lastWOLs(int idx)

Get information about the last WOL commands send to the machine. 
Index 0 is the latest one.

int

maxIPCount()

The maximum number of possible IPs for lastIPs and lastIPDate.

This is NOT the current count of valid values stored. You have to check if the returned values are valid.

int

maxWOLCount()

The maximum number of possible _Client_WOLInfo for lastWOLs.

This is NOT the current count of valid values stored.  You have to check if the returned values are valid.



Class _Client_WOLInfo


Class _Client_WOLInfo  parameter:

Return type

Name

Description

bool

success

Was the WOL command successfully?

uint8

bootTime

Seconds it took the machine from the WOL command to the rrClient being online.

rrTime

dateTime

Date and time of the WOL command.




Class _AppLicenses_v8

This class loads the render license count settings as set in rrConfig tab Render Lic.



Class _AppLicenses_v8 functions:

Return type

Name

Description

int

getCount()

There are count() number of render apps with license settings in _AppLicenses_v8.

_AppLicenses_v8_licItem

getLicense(in index)

Get license count settings at index.

bool

loadFromFile (string filename)

Loads the render license config file "filename". Returns True on success.

bool

saveToFile (string filename)

Saves the render license config file "filename". Returns True on success.



Class _AppLicenses_v8_licItem


Each render app license setting _AppLicenses_v8_licItem contains settings for each region.


The regions are:

Index

Name

0

Default

1

Workstation

2

Render Farm

3

Cloud




Class _AppLicenses_v8_licItem functions:

Return type

Name

Description

string

getRenderAppName()

Name of the render app. E.g. Maya, Nuke, Arnold

string

getPluginName()

Plugin name of this license. E.g. Yeti, Sapphire

int

getVersion()

Render licenses can be set differently for different versions of the render app. This returns the version this license count applies to.

int

getLicCount (int region)

License count

bool

getIsTypeSet (int region)

If a region is not set, then RR uses the count in "Default".

bool

getIsUnlimited (int region)

If the region has unlimited licenses, the lic count is ignored.

float

getCost (int region)

Costs that are used for RR to calculate the costs of a job.


SetRenderAppName (string name)



setPluginName(string name)



setVersion (int version)



setLicCount (int region, int count)



setIsTypeSet (int region,  bool isSet)



setIsUnlimited (int region,  bool IsUnlimited)



setCost (int region,  float cost)