collectors.storage — Advanced storage backends¶

collectors.storage.excel — Store data in Excel sheets¶

If your colleagues use MS Excel and require your data in that format, you want to use this storage backend. ;-)

class collectors.storage.excel.Excel(book, sheet)¶

Bases: collectors.core.BaseStorage

Stores the collectors data in the Excel file/workbook book within the sheet named sheet (sheet may also be an existing Worksheet).

Example:

import xlwt
from collectors import Collector, get, storage
class Spam(object):
    a = 1
spam = Spam()
book = xlwt.Workbook('my_data.xls')
collector = Collector(get(spam, 'a'),
        backend=storage.Excel(book, 'my_sheet'))

create_series(name, index)¶

Create a new ExcelSeries in the current sheed with the specified name at the position index.

class collectors.storage.excel.ExcelSeries(sheet, col_name, col_index)¶

Bases: object

An instance of this class will be create for each series. It provides the append() function for Excel.

It manages a column named col_name at index col_index with in an excel sheet. You don’t need to use this class directly.

append(value)¶

Append value to this series/sheet column.

collectors.storage.pytables — Store data in HDF5 files¶

PyTables can efficiently and easily cope with very large amounts of data. If you are planing to collect data from e.g. very long running simulations, you should consider using the Pytables storage instead of the default one to save memory.

class collectors.storage.pytables.PyTables(h5file, group, dtypes)¶

Bases: collectors.core.BaseStorage

This storage stores a collector’s series as an array in a HDF5 file. The series of a collector instance will be grouped together, so they can be addressed like /group_name/variable_name in the HDF5 file hierarchy.

Parameters:	h5file – An opend HDF5 as returned by tables.openFile() group – A name for the group or an existing group instance dtypes – A list of data types for the series ordered in the same way as the name-func-tuples.

PyTables is very tightly coupled with NumPy and requires you to specify the data type for each series. You can use NumPy’s type names for that. If you call the read() for a series, you’ll get a NumPy array containing all values for that series.

Note

This backend does NO automatic flushes. PyTables does that only when you close the HDF5 file, but you can call my_h5file.flush() anytime you want.

Example:

import tables
from collectors import Collector, get, storage
class Spam(object):
    a = 1
    b = 2.0
spam = Spam()
h5file = tables.openFile('spam.h5', mode='w')
collector = Collector(get(spam, 'a', 'b'),
        backend=storage.PyTables(h5file, 'spam_group', ('int', 'float'))
)

create_series(name, index)¶

Create a new EArray in the collector’s group named named with the type that can be found at index in the data types list passed to __init__.