DataStage基础培训教程

1、DataStageDataStage基础培训基础培训 JerryJerry2议程议程 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAd

2、ministrator & Manager Routine & ControlRoutine & Control3演示：演示：Hello WorldExtractTransformLoan4 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameters & TablesDefine Parameters & Tables Hash FileHash File、TransformerTransformer、AggregatorAggregator Direc

3、tor & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control议程议程5DataStage ArchitectureTarget(Database or File)ODBC/NativeDataStage Connect APIDataStage Server (WinNT, Win2000 or UNIX)ODBC/NativeData Sources (Database or Fi

4、le)DataStage Connect APIDataStage Connect APIDataStage Connect APIData flowData flow6DataStage ComponentsManager DesignerDirectorMetadata collection and management Design process flowRun jobs, check logs and set schedulesDataStageAdministratorCreate, Edit projects7 Hello WorldHello World DataStageDa

5、taStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control议程议程8全局变量

6、与全局变量与Job变量变量 全局变量全局变量- - 生命周期：整个生命周期：整个ProjectProject- - 在在AdministratorAdministrator中定义中定义 JobJob变量变量- - 生命周期：一个生命周期：一个JobJob- - 在在DesignerDesigner、ManagerManager中定义中定义9演示：定义一个演示：定义一个Job变量变量在在DesignerDesigner中定义参数中定义参数10Meta data definition 元数据管理的重要组成部分元数据管理的重要组成部分 在在MangerManger或或DesignDesign中中定义

7、定义 演示：演示：- import from a flat file in .txt format- import from a flat file in .txt format- import from an DBMS table- import from an DBMS table11演示演示Table Definition在在ManageManage中定义中定义TableTable12 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Para

8、meter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control议程议程13演示：生成事实表演示：生成事实表明细表事实表关联聚合14Hash File 用途：用途：- - 左连接时用作副表左连接时用作副表- - 多次被

9、访问的数据集多次被访问的数据集- - 存储其他临时数据存储其他临时数据 关键点：关键点：- - 必须指定必须指定keykey- output- output的的positionposition必须与必须与inputinput一致一致15Transformer 用途：用途：- - 提供丰富的运算符和函数提供丰富的运算符和函数- - 数据清洗、转换数据清洗、转换- - 关联多个数据源关联多个数据源 关键点：关键点：- - 副表的副表的keykey必须被主表的某个字段关联必须被主表的某个字段关联- - 尽量避免两个尽量避免两个TransformerTransformer直接相连直接相连16Aggre

10、gator 用途：用途：- - Sum, Max, Min, AverageSum, Max, Min, Average等聚合函数等聚合函数- - 一般用于生成事实表一般用于生成事实表17 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & Monitor

11、Director & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control议程议程18Debug and TuningView Status and LogsView Status and Logs- status, log, detail- status, log, detail等等多种视图多种视图- - 配合配合MonitorMonitor来查错、来查错、调优调优19Job Status Not CompiledNot Com

12、piled CompiledCompiled ResetReset RunningRunning FinishedFinished Finished (with warning)Finished (with warning) AbortAbort20ScheduleJob Job Add to ScheduleAdd to Schedule21 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash F

13、ileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control议程议程22Administrator Add a new projectAdd a new project Modify project propertiesModify project proper

14、ties- - 字符集字符集- - 日志保留天数日志保留天数- hash file and write catch - hash file and write catch Define environment viableDefine environment viable23Manager Import and export projects or jobsImport and export projects or jobs- - 两种文件格式：两种文件格式：.dsx .xml.dsx .xml- - 整个整个project, project, 根据根据categorycategory Tab

15、le definitionTable definition Manage RoutineManage Routine24演示：备份演示：备份project25 Hello WorldHello World DataStageDataStage ComponentsComponents Define Parameter & TableDefine Parameter & Table Hash FileHash File、TransformerTransformer、AggregatorAggregator Director & MonitorDirector & Monitor Administrator & ManagerAdministrator & Manager Routine & ControlRoutine & Control议程议程26Routine 一种自定义函数，使用一种自定义函数，使用VBVB