Real Time Delta Extraction Based on Triggers to Support Data Warehousing

被引:7
|
作者
Valencio, Carlos Roberto [1 ]
Marioto, Matheus Henrique [1 ]
Donega Zafalon, Geraldo Francisco [1 ]
Machado, Jose Marcio [1 ]
Momente, Julio Cesar [1 ]
机构
[1] Sao Paulo State Univ, Dept Ciencias Computacao & Estat, Sao Paulo, Brazil
关键词
data warehouse; ETL (Extract Transform and Load); CDC (Change Data Capture); trigger; automatic code generation;
D O I
10.1109/PDCAT.2013.52
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Nowadays large corporations require integrated data from diverse sources, leading to the use of data warehouse architectures for this purpose. To bypass problems related to the use of computational resources to process large volumes of data, an ETL (Extract, Transform and Load) technique with zero latency can be used, that works by constantly processing small data loads. Among the extraction techniques of the zero latency ETL are the use of logs, triggers, materialized views and timestamps. This paper proposes a structure capable of performing this task by means of triggers and a tool developed for the automatic generation of the SQL (Structured Query Language) code to create these trigger, besides showing its performance and comparing it to other techniques. Said method is relevant for the extraction of portions of selected information as it permits to combine conventional and real time ETL techniques.
引用
收藏
页码:293 / 297
页数:5
相关论文
共 50 条
  • [21] Optimizing Semi-Stream CACHEJOIN for Near-Real- Time Data Warehousing
    Naeem, M. Asif
    Mehmood, Erum
    Abbas, M. G.
    Jamil, Noreen
    [J]. JOURNAL OF DATABASE MANAGEMENT, 2020, 31 (01) : 20 - 37
  • [22] Real-time personalized commercial services using Data Warehousing and RFID technology
    Salguero, A.
    Araque, F.
    Carrasco, R.
    [J]. PROCEEDINGS OF THE 1ST RFID EURASIA CONFERENCE, 2007, : 284 - +
  • [23] Near Real-Time Data Warehousing with Multi-stage Trickle and Flip
    Zuters, Janis
    [J]. PERSPECTIVES IN BUSINESS INFORMATICS RESEARCH, 2011, 90 : 73 - 82
  • [24] A rewrite/merge approach for supporting real-time data warehousing via lightweight data integration
    Cuzzocrea, Alfredo
    Ferreira, Nickerson
    Furtado, Pedro
    [J]. JOURNAL OF SUPERCOMPUTING, 2020, 76 (05): : 3898 - 3922
  • [25] Integrated decision support systems: A data warehousing perspective
    March, Salvatore T.
    Hevner, Alan R.
    [J]. DECISION SUPPORT SYSTEMS, 2007, 43 (03) : 1031 - 1043
  • [26] Deadline-based scheduling in support of real-time data delivery
    Liu, Yanni Ellen
    Wong, Johnny W.
    [J]. COMPUTER NETWORKS, 2008, 52 (03) : 514 - 530
  • [27] Active data warehousing: A new breed of decision support
    Brobst, S
    [J]. 13TH INTERNATIONAL WORKSHOP ON DATABASE AND EXPERT SYSTEMS APPLICATIONS, PROCEEDINGS, 2002, : 769 - 769
  • [28] Near Real-Time Data Warehousing Using State-of-the-Art ETL Tools
    Joerg, Thomas
    Dessloch, Stefan
    [J]. ENABLING REAL-TIME BUSINESS INTELLIGENCE, 2010, 41 : 100 - 117
  • [29] Data warehousing: Improving time and cost management
    Ruffin, M
    [J]. M D COMPUTING, 2001, 18 (01): : 56 - +
  • [30] Integrating wireless technologies and data warehousing for distributed imaging based support in an orthodontic practice
    Mosca, R
    Revetria, R
    Briano, E
    Savastano, F
    [J]. PROCEEDINGS OF THE SECOND IASTED INTERNATIONAL CONFERENCE ON BIOMEDICAL ENGINEERING, 2004, : 426 - 430