Databases Reference
In-Depth Information
One thing that we cannot do is add extra text into the FORMAT attribute. For instance,
in the second case, it would be nice to use a FORMAT attribute such as this one:
FORMAT = $1_id::$2
Unfortunately, this will not function as we hope and will instead create the field id .
Chaining transforms
As covered before in the Attributes with class section, transforms are executed in a
particular order. In most cases, this order does not matter, but there are occasions
when you might want to chain transforms together, with one transform relying on a
field created by a previous transform.
A good example is the source flattening that we used previously, in the Overriding
source section. If this transform happened before our transform in the Creating a
session field from source section, our session field would always have the value x .
Let's reuse two transforms from previous sections and then create one more
transform. We will chain them to pull the first part of session into yet another
field. Say we have these transforms:
[myapp_session]
SOURCE_KEY = MetaData:Source
REGEX = session_(.*?)\.log
FORMAT = session::$1
WRITE_META = True
[myapp_flatten_source]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = (.*session_).*.log
FORMAT = source::$1x.log
[session_type]
SOURCE_KEY = session
REGEX = (.*?)-
FORMAT = session_type::$1
WRITE_META = True
To ensure that these transforms run in order, the simplest thing would be to place
them in a single TRANSFORMS attribute in props.conf , like so:
[source:*session_*.log]
TRANSFORMS-s = myapp_session,myapp_flatten_source,session_type
 
Search WWH ::




Custom Search