|
|
| (86 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | BIRD System (BIRD): Biological Integration and Retrieval Data was designed by Hoan Nguyen at LBGI laboratory (POCH Team) of IGBMC[http://www-igbmc.u-strasbg.fr] Strasbourg | + | BIRD System : Biological Integration and Retrieval Data was designed by Hoan Nguyen at LBGI laboratory (POCH Team) of IGBMC[http://www-igbmc.u-strasbg.fr] Strasbourg |
| − | ==What is BIRD System== | + | ==What is the BIRD System== |
| − | ===Scientific Context===
| |
| − | Since 2000, thanks to the availability of the human genome and the rapid progress of biotechnologies and information technologies, numerous large biomedical datasets have been generated. Thus, modern biomedical information corresponds to a high volume of heterogeneous data that doubles in size every year (Statistics NCBI) and that covers very different data types, including patient data (from phenotypic, environmental or behavioral origins), gene data (including genome environment, gene expression status, enzymatic activity, gene product modification…) and the processes, protocols or treatments used to generate the information. In this context, systemic approaches are now being developed to analyze and compare this huge amount of information, in order to identify genes and to predict their functions in the cascade of events and networks involved for example, in the emergence of a disease. This requires the development of dynamic and powerful systems to store, assemble, integrate and process very large datasets from different sources. Recently, the Decrypthon initiative (Decrypthon), resulting from a collaboration between AFM/CNRS/IBM, has been instigated, firstly to develop a computing grid that connects hundreds of processors installed in various data-processing centres of French universities and, secondly to provide a facilitated access to the data for the scientific biological community. In the framework of the Decrypthon initiative, several biomedical projects are in progress requiring on the one hand, a strong computational capacity and on the other hand, the deployment in the grid environment of a data integration system able to manage automatically large volumes of heterogeneous data and to quickly process complex queries and versioning management.
| |
| − | | |
| | ===BIRD System Overview=== | | ===BIRD System Overview=== |
| − | BIRD System (Nguyen et al, CORIA 2008, Hermes Edition) was designed to manage large collections of biological data and to intensive computation and simulation. BIRD heritages somes main idea of Saada project[http://amwdb.u-strasbg.fr/saada/article.php3?id_article=32]. A generic configurable data model has been designed and allows the simultaneous integration of genomics, transcriptomics and ontology datasets using a limited number of product mapping rules provided by the user (operator or system administrator). The integration rules allow the easy creation of the database according to semantic topics and real requirements. | + | The BIRD System was designed to manage large collections of biological data ([[Bird_Databases_List]]) and to perform intensive computation and simulation. BIRD has inherited some of the idealogy of the Saada project [http://amwdb.u-strasbg.fr/saada/article.php3?id_article=32]. A generic configurable data model has been designed and allows the simultaneous integration of genomics, transcriptomics and ontology datasets using a limited number of product mapping rules provided by the user (operator or system administrator). The integration rules allow the easy creation of a database according to semantic topics and real requirements. |
| − | BIRD is driven with a high level query engine, based on SQL and a full text engine allowing the biologist to quickly extract knowledge without programming. Thanks to such an engine, the system is capable to generate the sub-bank of data in accordance with the real requirement. | + | BIRD is driven by a high level query engine (BIRD-QL), based on SQL and a full text engine allowing the biologist to quickly extract knowledge without programming. Thanks to such an engine, the system is capable of generating sub-databases in accordance with the real requirements of a given project. |
| | | | |
| | The hosted data can be accessed by the community using various methods such as a Web interface, Http Service, an API Java or a BIRD-QL Engine Query. | | The hosted data can be accessed by the community using various methods such as a Web interface, Http Service, an API Java or a BIRD-QL Engine Query. |
| | | | |
| − | BIRD System is developed with the Java technology. BIRD System uses IBM DB2 for data server; Websphere Federtion Server for virtual databases. The web application is hosted by a Tomcat Server or by a WebSphere Application Server. | + | The BIRD System is developed using the Java technology and uses the IBM DB2 as the data server, as well as the Websphere Federation Server for virtual databases. The web application is hosted either by a Tomcat Server or by a WebSphere Application Server. |
| − | | |
| − | BIRD System is not only a retrieval data system but also a plate-forme of Kownlegde Discovery in Biological Database. We use IBM Miner Intelligent (association rules, classification, ..) in order to develop the data mining model.
| |
| − | | |
| − | | |
| − | The first goal of Bird System is to implementation of the Décrypthon Data Center [http://bird.u-strasbg.fr:9080/BirdSystem/HomePage.do] [http://decrypthon-1.ens-lyon.fr:9080/BirdSystem/HomePage.do] in the framework of Décrypthon Programme (AFM/CNRS/IBM ) [http://www.decrypthon.fr]
| |
| | | | |
| − | ==Data Format & DATABASES List ==
| + | The BIRD System is not only a data retrieval tool, but also provides a platform for Knowledge Discovery in Biological Databases or an inductive database. We use the IBM Intelligent Miner (association rules, classification, ..) in order to develop the data mining model. The user can then use BIRD-QL for mining pertinent information or for analyzing the relational patterns based on the descriptive patterns available in the BIRD-QL engine. |
| | | | |
| − | Format: EMBL, GENBANK, XML, CSV, OBO/OWL, PDB, UMD, Relational Schema (XML Metadata).
| |
| | | | |
| − | Databases: GENBANK, EST, WGS, REFSEQ, PDB, UNIPROT, UCSC, INTERPRO, GO, TAXONOMY, MACSIM, EVI-GENORET (local user), STRING (local user), UMD Data (local user), ...
| + | The first goal of the Bird System is the implementation of the Décrypthon Data Center in the framework of the Décrypthon Programme (AFM/CNRS/IBM ) [http://www.decrypthon.fr] |
| | | | |
| | ==[[BIRDQL]] Biological Query Language == | | ==[[BIRDQL]] Biological Query Language == |
| − | ===BIRDQL in few words===
| |
| − | The heterogeneous data integrated in BIRD System are represented by several relational tables. The exploitation of these data by SQL queries is not obvious except for developers or computer scientist experts.
| |
| − |
| |
| − | Building queries with SQL in this context is not easy with because that requires to use joins (terme technique) to select data in multiple tables. This complexity must be hidden by HTML forms but a lot of queries can not be setup with HTML forms.
| |
| − |
| |
| − | We proposes own query language (BIRDQL), there is new standard biological query language allowing the biologist or clinician to create data retrieval protocols without exhaustive knowledge of the data sources and their architecture. BIRD System is driven with a high level query engine: BIRDQL, which makes it possible for biologists to express easily queries and to extract knowledge by classical constraints and scientific functions (StructuralDistance,SequencePattern,AssociationRule...).
| |
| − |
| |
| − | BIRDQL in not a mathematically complete language but indeed an idiom adpated to the GUI, human readable enough to be modified by hand.
| |
| − |
| |
| − |
| |
| − | ==[[BIRD Data Access Protocol|Data Access Protocoles]]==
| |
| − |
| |
| − | ==BIRD business intelligence ==
| |
| − |
| |
| − | ===Theories and Functionalities===
| |
| − |
| |
| − | KDD Steps
| |
| − | [[Image:kddstep.jpg]]
| |
| − |
| |
| − | [[Image:kddsetp2.jpg]]
| |
| − |
| |
| − | KDD Tecnhique & Algorithm
| |
| − | [[Image:algo3.jpg]]
| |
| − |
| |
| − | KDD Data Model & View
| |
| − | [[Image:modelview.jpg]]
| |
| − |
| |
| − |
| |
| − | ====Association rule learning====
| |
| − | a.'''What Is Association Rule Mining?'''
| |
| − |
| |
| − | Describing association relationships among the attributes in the set of relevant data
| |
| − |
| |
| − | Frequent pattern mining: find all frequent patterns in a database
| |
| − |
| |
| − | Frequent patterns: patterns (set of items, sequence, etc.) that occur frequently in a database [AIS93]
| |
| − |
| |
| − | Frequent pattern mining: finding regularities in data
| |
| − |
| |
| − | +What products were often purchased together? Beer and diapers?!
| |
| − |
| |
| − | +What are the subsequent purchases after buying a product( ex. car)?
| |
| − |
| |
| − | +Can we automatically profile patient or gene ?
| |
| − |
| |
| − | b.'''Basic'''
| |
| − |
| |
| − | Rule Definition
| |
| − |
| |
| − | Body ==> Consequent [ Support , Confidence ]
| |
| − | (IF <> THEN <>)
| |
| − | Body: represents the examined data.
| |
| − | Consequent: represents a discovered property for the examined data.
| |
| − | Support: represents the percentage of the records satisfying the body or the consequent.
| |
| − | Confidence: represents the percentage of the records satisfying both the body and the
| |
| − | consequent to those satisfying only the body
| |
| − |
| |
| − |
| |
| − |
| |
| − |
| |
| − | Itemset: a set of items
| |
| − |
| |
| − | =>E.g., acm={a, c, m}
| |
| − |
| |
| − | Support of itemsets
| |
| − |
| |
| − | =>Sup(acm)=3
| |
| − |
| |
| − | Given min_sup=3, acm is a frequent pattern
| |
| − |
| |
| − | Frequent pattern mining: find all frequent patterns in a database
| |
| − |
| |
| − |
| |
| − | [[Image:rulesbasic.jpg]]
| |
| − |
| |
| − |
| |
| − |
| |
| − | c.'''Apriori Algorithm'''
| |
| − |
| |
| − |
| |
| − | Ck: Candidate itemset of size k
| |
| − | Lk : frequent itemset of size k
| |
| − |
| |
| − | L1 = {frequent items};
| |
| − | for (k = 1; Lk !=Q; k++) do
| |
| − | Ck+1 = candidates generated from Lk;
| |
| − | for each transaction t in database do increment the count of all candidates in Ck+1 that are
| |
| − | contained in t
| |
| − | Lk+1 = candidates in Ck+1 with min_support
| |
| − | return UkLk; (Union)
| |
| − |
| |
| − | [[Image:Apriori.jpg]]
| |
| − |
| |
| − | ====Kohonen´s feature maps====
| |
| − | A Kohonen’s self organizing feayture map (K-map) is uses analogy with such biological neural
| |
| − | structures where the placement of neurons is orderly and reflects structure of external (sensed)
| |
| − | stimuli (e.g. in auditory and visual pathways).
| |
| − |
| |
| − | K-map learns, when continuous-valued input vectors are presented to it without specifying the
| |
| − | desired output. The weights of connections can adjust to regularities in input. Large number of
| |
| − | examples is needed.
| |
| − |
| |
| − | K-map mimics well learning in biological neural structures. It is usable in speech recognizer
| |
| − |
| |
| − | This is a flat (two-dimensional) structure with connections between neighbors and connections
| |
| − | from each input node to all its output nodes.
| |
| − |
| |
| − | It learns clusters of input vectors without any help from teacher. Preserves closeness (topolgy).
| |
| − |
| |
| − | '''Learning in K-maps'''
| |
| − |
| |
| − | 1. Initialize weights to small random numbers and set initial radius of neighborhood of nodes.
| |
| − |
| |
| − | 2. Get an input x1, …, xn.
| |
| − |
| |
| − | 3. Compute distance dj to each output node:
| |
| − | dj = (xi - wij)2
| |
| − |
| |
| − | 4. Select output node s with minimal distance ds.
| |
| − |
| |
| − | 5. Update weights for the node s and all nodes in its neighborhood:
| |
| − | wij´= wij + h* (xi - wij), where h<1 is a gain that decreases in time.
| |
| − |
| |
| − | Repeat steps 2 - 5.
| |
| − |
| |
| − | ===DB2 Miner Intelligent (API)===
| |
| − |
| |
| − | Data flow of the mining procedure (FindDeviations ex.)
| |
| − | [[Image:kdd_model.jpg]]
| |
| − | Finding deviations
| |
| − |
| |
| − | Finding groups with similar characteristics (ClusterTable procedure)
| |
| − |
| |
| − | You can find groups with similar characteristics by using the ClusterTable procedure.
| |
| − | When to do it:
| |
| − | The database might contain patient data including demographic data, for example: v Gender v Age v
| |
| − | Profession v Family statusThe information might also include the income or the socio-demographic group of the customer
| |
| − |
| |
| − |
| |
| − | Finding relationships (FindRules procedure) You can find relationships in your data by using the FindRules procedure.
| |
| − |
| |
| − |
| |
| − | Predicting future behavior (PredictColumn procedure)
| |
| − |
| |
| − | In the tables or views of your database (Transciptomic or clinical Data), there might
| |
| − | be one column that you are particularly interested in. In the clinical data, you can find
| |
| − | relations between symptoms and diseases. With this information, you can predict the potential diseases of new patients
| |
| − |
| |
| − | Finding most important fields (FindMostImpFields procedure)
| |
| − |
| |
| − | You can find most important fields by using the FindMostImpFields procedure.
| |
| − |
| |
| − | ===Kownledge Discovery in Biological Database===
| |
| − |
| |
| − | Some questions ?
| |
| − |
| |
| − | � Can we perform sequence analysis in order to detect sequence patterns that
| |
| − | occur very often in the chromosome?
| |
| − |
| |
| − | � If a mutation takes place in a chromosome, does there exist any relationships
| |
| − | between nucleotides? If yes, does a mutation of the one nucleotid also
| |
| − | influence the other ones and can we use one of the techniques described
| |
| − | above to find such relationships?
| |
| − |
| |
| − | � If we translate the activities of the nucleotides into a frequency, can we then
| |
| − | detect similar sequences that occur over time? Can we then find indicators
| |
| − | that are probably responsible for mutation?
| |
| − |
| |
| − | � Sequence tagged site (STS) are a short (200 to 500 base pairs) DNA
| |
| − | sequence that has a single occurrence in the human genome. Can we detect
| |
| − | such STSs using KDD?
| |
| − |
| |
| − | � Genetic disorders resulting from the combined action of alleles of more than
| |
| − | one gene (for example, heart disease, diabetes, and some cancers). Although
| |
| − | such disorders are inherited, they depend on the simultaneous presence of
| |
| − | several alleles; therefore, the hereditary patterns are usually more complex
| |
| − | than those of single gene disorders. Can we detect such polygenic disorders
| |
| − | using KDD ?
| |
| − |
| |
| − | � A problem in bioinformatics is the determination of the order of the
| |
| − | nucleotides in a DNA molecule or the order of amino acids in a protein. This is
| |
| − | referenced as sequencing. Can we detect such poly genic disorders using KDD?
| |
| − |
| |
| − | ==BIRD System in Action ==
| |
| − | ===Décrypthon Data Center===
| |
| − |
| |
| − |
| |
| − | ====Overview====
| |
| − |
| |
| − | [[Image:ddc_idea.jpg]]
| |
| − |
| |
| − | BIRD System is core of Décrypthon Data Center.
| |
| − | Sharing of large scare biological data for applications (Macsim, MS2PH, Macgos, Ordali..)
| |
| − | runing on Décrypthon Grid.
| |
| − | Managing of generated data (result) on grid
| |
| − |
| |
| − | Sharing of data and services for scientific community
| |
| − | http://bird.u-strasbg.fr:9080/BirdSystem/HomePage.do
| |
| − |
| |
| − |
| |
| − | [[Image:bird_ddc.jpg]]
| |
| − |
| |
| − | ===Macsim uses BIRDQL engine===
| |
| − | MACSIMS:Multiple Alignment of Complete Sequences Information Management System (Thompson et al, 2006).MACSIMS provides a unique environment that facilitates knowledge extraction and the presentation of the most pertinent information to the biologist .
| |
| − |
| |
| − | Macsim gets direct connection with Bird database
| |
| − |
| |
| − | ===GPS uses BIRDQL engine===
| |
| − | http://nucleic.fr
| |
| − |
| |
| − | ===Gscope utilise BIRD===
| |
| − | Gscope peut se mettre maintenant en connexion directe avec Bird
| |
| − |
| |
| − |
| |
| − | * proc '''BirdFromQueryText''' {Texte {OutFile ""} {BirdUrl ""}}
| |
| − | * proc '''BirdFromQueryFile''' {Fichier {OutFile ""} {BirdUrl ""}}
| |
| − |
| |
| − | Bird sait intégrer les fiches infos d'un projet Gscope. On peut alors les interroger directement par http ou par Gscope ou, mieux, par des affiches avec la commande '''BirdGscopeSearch'''
| |
| − |
| |
| − | ==BIRD Development ==
| |
| − |
| |
| − | [[lbgiki:BIRD_implementation|BIRD Implementation]]
| |
| − |
| |
| − | ===Origin BIRD System===
| |
| − | BIRD was based on main principe of Saada project [http://amwdb.u-strasbg.fr/saada/article.php3?id_article=32].
| |
| − | SAADA - Systèm d’Archivage Automatique des Données Astronomiques
| |
| − | First Goal : Archive & Exploitation of Data of the European XMM Newton satellite [http://xcatdb.u-strasbg.fr/2xmm/home].The 2XMM catalogue of X-ray sources,
| |
| − | the largest of its kind ever, has now been released.
| |
| − | In a PhD Framework (2002-2005, Prototype Saada V.1.3) of Dr.NGUYEN at University of Strasbourg I,Supported by the
| |
| − | CNES[www.cnes.fr] and the Alsace Region, Supervised by Dr. Michel and Dr.Motch.
| |
| − |
| |
| − |
| |
| − | [[Image:saada_bird.jpg]]
| |
| − |
| |
| − | ===Conceptual Data Model===
| |
| − |
| |
| − | In order to automatically integrate heterogeneous data, we have designed several business data model corresponding to the real format of the data banks. Figure below illustrates the conceptual data model of the BIRD system. It can simultaneously host several bank types. Each type can itself concern several User Defined banks having the same format. Thanks to this conceptual model, BIRD can host different versions of a given data bank and manage them so that the programs launched on grid computing within an application can exploit the same data version during their computation time.
| |
| − |
| |
| − | [[Image:model1.jpg]]
| |
| − |
| |
| − |
| |
| − |
| |
| − | The data model of a data bank is predefined in an XML configuration file . This metadata is used to create the Java and SQL code. The code generation is launched at the moment of the configuration of the BIRD data bank or at data loading or reloading. In the example given figure 3, the metadata of Genbank are used to create Genbank-EST, Genbank-Refseq.
| |
| − |
| |
| − | [[Image:model2.jpg]]
| |
| − |
| |
| − |
| |
| − | This Figure illustrates the business model for Genbank. Each bank can have several associated entries. Each entry has its associated information like Dbref, SEQData, FTSource,... According to our design, the Java classes of the business models are automatically generated by BIRD. Only instances of classes inherited from super ObjectPersistence will be recognized by the BIRD API. This super class contains common attributes and methods for all generated classes.
| |
| − |
| |
| − | ===Query Engine===
| |
| − |
| |
| − | ===Data Integration===
| |
| − |
| |
| − | [[Image:integration1.jpg]]
| |
| − |
| |
| − | The creation of a database goes through some principal stages :
| |
| − | Initially the relational schema system (meta-model) is created when BIRD is installed.
| |
| − | In the second phase, the configuration module creates the business data model including SQL and Java codes corresponding to the predefined metadata given by the XML configuration. Then, the system analyzes some integration rules to select the data files and proceed to their conversion and loading into the relational tables .
| |
| − |
| |
| − | ===Architecture===
| |
| − | [[Image:bird_arch.jpg]]
| |
| − |
| |
| − | ===Key Technologies===
| |
| − | Relational Core store
| |
| − |
| |
| − | IBM DB2 WareHouse V9.1
| |
| − | WebSphere Federation Server
| |
| − |
| |
| − | WEB Server & Services
| |
| − |
| |
| − | IBM WebSphere Application Server ( main Portal)
| |
| − | Tomcat Server (services, non graphic)
| |
| − | Hibernate and JSF-Java Server Face
| |
| − | Object Relational Mapping
| |
| − | Web component
| |
| − |
| |
| − | XML & JAVA
| |
| − |
| |
| − | ===Project Distribution===
| |
| − | Not net to public
| |
| − |
| |
| − | ==Publications==
| |
| − |
| |
| − | To cite BIRD System, please use the following publication;
| |
| | | | |
| − | 1. Nguyen H., Berthommier G., Friedrich A., Poidevin L. ,Ripp R. , Moulinier L. and Poch O. Introduction du nouveau centre de données biomédicales Décrypthon, CORIA 2008.
| + | The heterogeneous data integrated in the BIRD System are represented by several relational tables. The exploitation of these data by SQL queries is not obvious and can only be performed by expert developers or computer scientists. |
| | | | |
| − | 2. "Conception of the BIRD System" is preparing for .....
| + | In this context, building complex queries with SQL involves the use of joins (technical term) to select data in multiple tables. This complexity can be hidden by HTML forms, but many types of queries cannot be specified with HTML forms. |
| | | | |
| − | 3. "BIRDQL-A new Biological Query Language " is preparing for....
| + | We have therefore developed our own query language ([[BIRDQL]]), which is a new biological query language that allows the biologist or clinician to create data retrieval protocols without requiring exhaustive knowledge of the data sources and their architecture. BIRDQL makes it possible for biologists to easily express queries and to extract knowledge using classical constraints and scientific functions (StructuralDistance,SequencePattern,AssociationRule...). |
| | | | |
| | + | [[BIRDQL]] in not a mathematically complete language but instead is an idiom that is adapted to the GUI and is human readable enough to be modified by hand. |
| | + | see more [[BIRDQL]] |
| | | | |
| | | | |
| − | ==Contact==
| |
| − | Nguyen Ngoc Hoan,PhD
| |
| − | IGBMC Strasbourg
| |
| − | 1 rue Laurent Fries
| |
| − | BP 10142
| |
| − | 67404 Illkirch CEDEX / France
| |
| − | Mail:[mailto:nguyen@igbmc.fr nguyen@igbmc.fr]
| |
| − | Tel: 0033 388653302
| |
| − | --[[User:Nguyen|Nguyen]] 15:07, 16 February 2008 (CET)---
| |
| | | | |
| − | ==FAQ?==
| + | [[Category:Bird_project]] |
