Background: The dataset produced is only for the last 30 days of data, but we need to keep the data for much longer.
Solution: use dataflow/recipe to store the trend in a new dataset
1. Download the dataset in a CSV file and load it as a "trend" dataset
2. Use dataflow/recipe to read from the original and trend dataset
3. Append both datasets
4. Use computeRelative node (dataflow) or "Multiple row formula" transformation (recipe), Partition By the key field, this is to mark the 1st record
5. Filter out rows not marked as 1st record
6. Done
Recipe
No comments:
Post a Comment