Databases Reference
In-Depth Information
>>>
dset
.
attrs
[
'ones'
]
=
np
.
ones
((
100
,))
>>>
dset
.
attrs
[
'ones'
]
array([ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1.])
There are limits, though. In HDF5, with the default settings (“compact” storage, as
opposed to “dense” storage), attributes are limited to a size of 64k. For example, if we
try to store a (100, 100) array, it complains:
>>>
dset
.
attrs
[
'ones'
]
=
np
.
ones
((
100
,
100
))
ValueError: unable to create attribute (Attribute: Unable to initialize object)
Most regrettably, we discover that in this case the previous attribute was wiped out:
>>>
dset
.
attrs
[
'ones'
]
KeyError: "can't open attribute (Attribute: Can't open object)"
This is one of the (very few) cases where h5py's interaction with the
file is not atomic. Exercise caution with larger array attributes.
One way around this limitation is simply to store the data in a dataset, and link to it
with an object reference (see
Chapter 8
):
>>>
ones_dset
=
f
.
create_dataset
(
'ones_data'
,
data
=
np
.
ones
((
100
,
100
)))
>>>
dset
.
attrs
[
'ones'
]
=
ones_dset
.
ref
>>>
dset
.
attrs
[
'ones'
]
<HDF5 object reference>
To access the data, use the reference to retrieve the dataset and read it out:
>>>
ones_dset
=
f
[
dset
.
attrs
[
'ones'
]]
>>>
ones_dset
[
...
]
array([[ 1., 1., 1., ..., 1., 1., 1.],
[ 1., 1., 1., ..., 1., 1., 1.],
[ 1., 1., 1., ..., 1., 1., 1.],
...,
[ 1., 1., 1., ..., 1., 1., 1.],
[ 1., 1., 1., ..., 1., 1., 1.],
[ 1., 1., 1., ..., 1., 1., 1.]])
Strings and File Compatibility
There are a couple of types that have special handling. First, there is a subtle difference
in HDF5 regarding the type of a string. In the previous example, we assigned a Python
