ADF: Data Flow: how to filter data

( Collapse )
SQL Server Integration Services (SSRS) is a part of SLQ Server that allows to build various reports.
The SSRS is a separate Windows Service (EXE file) on an MS SQL Server and should be started separately:
A SSRS report is contained in a special file format RDL. Here is an example of an RDL file (during the design stage; not on runtime stage):
Here is an example of design of an RDL file opened in Visual Studio:
- The “1” is deign of the file;
- The “2” is a simple status text within the file;
- The “3” is Tablix control embedded into the Report. It is connected to a Data Source that is denoted by the number “4”. Therefore, at runtime the “3” Tablix man have many rows (that may spread out on many pages) brought from a DB that the “4” refers. The “4” may be either simple SQL query or a call of a huge and sophisticated SQL procedure.
The Reports may be parametrized that improves their flexibility.
Ans inctance of an SSRS often has an URK like http://<SQL Server hostname>/Reports .
This task is described in the following Udemy lesson: https://www.udemy.com/course/learn-azure-data-factory-from-scratch/learn/lecture/23931350#overview :
We already have a set of streams; no we have to finish it with final “sink” stream.
See more in Word file at https://docs.google.com/document/d/1mFTNMsZ9RJcOz5ilXcApZcTGoMtfPIzJ/edit?usp=sharing&ouid=106961909928818620244&rtpof=true&sd=true
.
A problem: we have a Data Flow; and we want to leave only rows where:
Column continent is equal to 'Europe'
And column country_code is not NULL
To do this, add a filtering operation into the Data Flow:
A problem: I have a collection of data stored in a JSON file:
…
[
{
"sourceRelativeURL":"cloudboxacademy/cov
"sinkFileName":"cases_deaths.csv"
},
{
"sourceRelativeURL":"cloudboxacademy/cov
"sinkFileName":"hospital_admissions.csv"
}
]
…
A problem: I have an ADF Pipeline. I would like that the Pipeline was automatically started every 15 minutes starting with a date Start Date and till a End Date.
A solution: create a Trigger of type Tumbling window:
(