Managing ETL dependencies with SAP BusinessObjects Data Services (Part 4)
Are you glad with
the approach you presently manage
the dependencies in your ETL? within the talked concerning the options I’m
expecting from a dependency management system, and what ar the
most prospects offered
(directly or indirectly) by knowledge Services.
In projected Associate
in Nursing design (structure
and expected behavior) for a dependency management system within knowledge Services
I explained the entire implementation
details. during this final half, I’m about to offer
you a
feedback on however it
went “in real life” additionally as potential enhancements.
So, the
foremost necessary question
is after all “Does
it work?”. I’m happy to verify that affirmative, it does! And it makes quite
an distinction within
the lifetime
of the
customer’s knowledge Services
administrator. after all it
doesn’t modification something if the ETL
is running fine, however once there ar issues, this dependency management system are
often an
enormous time-saver!
In his atmosphere, our client had presently sixty
four flows,
with eighty six dependencies
between them (this is adequate the
amount of rows within
the FLOW_DEPENDENCIES
table). Let’s take a real-life example: due to a
brand new exception, miscalculation happened within
the flow chargeable
for the client dimension.
As a consequence, five relying flows
(which inhabited truth tables) weren't dead. What would are future step for
the administrator when fixing
the exception? while not the
dependency management system, he would have to be compelled to either
re-run the full ETL
(including the fifty eight flows that had
already been dead successfully),
or to form a
brand new job with simply the six flows, concentrate to the
variables, etc. however since
he currently has
the dependency management system, the administrator merely re-runs the
work by giving
the ID of the last job, and every one the
flows that were with success dead ar merely skipped.
All in all, the
time savings and peace of mind ar extremely appreciated
by the client.
Now, will we
tend to improve
this system? Definitely! two potential improvements:
• Sending Associate
in Nursing email to
the administrator at the top of the
work with a outline of what
happened. this will be
done at intervals a
script at the top of the
work. The image below
shows you Associate in Nursing example.
You’ll notice that I’ve enclosed samples
of SQL queries to create it
easier for the administrator to research what
happened.
• It’s alittle outside
the scope of this text, however you
may use the
FLOW_STATUS table to form a
report concerning the
technical standing of
the business processes. This report would enable finish users to
simply see if the
information they
have has been
loaded these days. you'll be able to see
below Associate in Nursing example
of such a business method standing report.
There’s a final
question I’ve not talked concerning yet: however can
we organize
the flows within the job? once
I used this
dependency management system for the primary time,
I started by extracting knowledge from
the supply systems,
followed by all the dimension tables, and finished with all the
actual fact tables. these
days i'd do things otherwise, supported a
presentation created by
Werner Daehn (a SAP Product Manager), i'd produce one work
flow per business method. for instance one
work flow for the overall Ledger
business method, with all the flows required for the
information extraction,
dimensions and truth tables. the
work would
execute the work flow for the business method one, then for the business method two, etc. This organization permits simple freelance testing of
every business method. however here you
will think:
“Ok, however what concerning dimensions/extractions that ar employed
in multiple
business processes? can the
work run them
multiple times?” That downside are
going to be avoided by
checking the choice “Execute solely once” for
every involved work flow.
I have currently reached the
ultimate step my ETL
dependency management system. What does one think
about it? Any
questions? have you ever tried
it by yourself? Let ME recognize by posting
a comment below.
Read more at http://smallseotools.com/article-rewriter/#hj71EhSkDYIk1hor.99
So, the foremost necessary question is after all “Does it work?”. I’m happy to verify that affirmative, it does! And it makes quite an distinction within the lifetime of the customer’s knowledge Services administrator. after all it doesn’t modification something if the ETL is running fine, however once there ar issues, this dependency management system are often an enormous time-saver!
In his atmosphere, our client had presently sixty four flows, with eighty six dependencies between them (this is adequate the amount of rows within the FLOW_DEPENDENCIES table). Let’s take a real-life example: due to a brand new exception, miscalculation happened within the flow chargeable for the client dimension. As a consequence, five relying flows (which inhabited truth tables) weren't dead. What would are future step for the administrator when fixing the exception? while not the dependency management system, he would have to be compelled to either re-run the full ETL (including the fifty eight flows that had already been dead successfully), or to form a brand new job with simply the six flows, concentrate to the variables, etc. however since he currently has the dependency management system, the administrator merely re-runs the work by giving the ID of the last job, and every one the flows that were with success dead ar merely skipped.
All in all, the time savings and peace of mind ar extremely appreciated by the client.
Now, will we tend to improve this system? Definitely! two potential improvements:
• Sending Associate in Nursing email to the administrator at the top of the work with a outline of what happened. this will be done at intervals a script at the top of the work. The image below shows you Associate in Nursing example. You’ll notice that I’ve enclosed samples of SQL queries to create it easier for the administrator to research what happened.
• It’s alittle outside the scope of this text, however you may use the FLOW_STATUS table to form a report concerning the technical standing of the business processes. This report would enable finish users to simply see if the information they have has been loaded these days. you'll be able to see below Associate in Nursing example of such a business method standing report.
There’s a final question I’ve not talked concerning yet: however can we organize the flows within the job? once I used this dependency management system for the primary time, I started by extracting knowledge from the supply systems, followed by all the dimension tables, and finished with all the actual fact tables. these days i'd do things otherwise, supported a presentation created by Werner Daehn (a SAP Product Manager), i'd produce one work flow per business method. for instance one work flow for the overall Ledger business method, with all the flows required for the information extraction, dimensions and truth tables. the work would execute the work flow for the business method one, then for the business method two, etc. This organization permits simple freelance testing of every business method. however here you will think: “Ok, however what concerning dimensions/extractions that ar employed in multiple business processes? can the work run them multiple times?” That downside are going to be avoided by checking the choice “Execute solely once” for every involved work flow.
I have currently reached the ultimate step my ETL dependency management system. What does one think about it? Any questions? have you ever tried it by yourself? Let ME recognize by posting a comment below.
Read more at http://smallseotools.com/article-rewriter/#hj71EhSkDYIk1hor.99
Seneca Software Solutions
3651 Buckley st,
Apt 905Santa Clara,
California 95051,
Seneca Software Solutions
3651 Buckley st,
Apt 905Santa Clara,
California 95051,
Sam:630-444-7493 ,
Email:Sam@senecatraining.com
info@senecatraning.com
Sam:630-444-7493 ,
Email:Sam@senecatraining.com
info@senecatraning.com
Seneca Software Solutions
3651 Buckley st,
Apt 905Santa Clara,
California 95051,
Sam:630-444-7493 ,
Email:Sam@senecatraining.com
info@senecatraning.com
