Tuesday, April 23, 2013

BI and Predictable analysis using Open Source BI Tool



Gartner defines business intelligence (BI) as an umbrella term that spans the people, processes and applications/tools to organize information, enable access to it and analyze it to improve decisions and manage performance. Today’s banking world is changing; it is changing as customer is changing dynamically. For years banks have relied on internal insight they have on their customer to manage risk fraud and also decide on product development, marketing strategy and customer communication.  Banking has also changed from personalized to virtual banking where customers are more relying on various online channels like ATM, Internet etc rather than visiting tellers or relationship managers. However the availability of vast pool of structured and unstructured “BIG DATA” allows for real time analysis and decision making that can help save money and also increase revenue.
“There were 5 exabytes of information created by the entire world between the dawn of civilization and 2003. Now that same amount is created every two days.”
– Eric Schmidt, CEO/Google
With majority of the Tier 1 and Tier 2 Banks have fully implemented IT system  there is vast pool of customer and transactional data is available in the central system for analysis. BIG Data has become a new buzz word for most of the Banking Product Companies who are seeing depleting revenues in CBS sales. There is lot of white papers available on the web with reference to new products, tool for BI and DW with new jargon for Big Data. There are also lots of white paper for comparative analysis of Open Source and Proprietary system sponsored by Big Players with emphasis on why not use Open Source. Open source solutions has always been an earning opportunity to the big proprietary players and failure to large bank that have tried implementing open source solutions themselves, but have always used product built by proprietary player using open source.  Lack of proper documentation and highly technical implementation meant only for the Geeks are one of the major reasons for not acceptance of open source solutions.
BI open source components available are
1.       Reporting : Use of open report systems like Jasper, BIRT etc
2.       MULTIDIMENSIONAL ANALYSIS (OLAP): Use of  Jpivot/Mondrian, JPalo/Mondrian, JPXMLA to provide multi dimensional cube and cube charts These tool allows the multidimensional analysis through flexible and user-friendly OLAP engines. Users can monitor the data on different detail levels and from different perspectives, through drill-down, drill-across, slice-and-dice, drill-through processes.


Charts :  JFreeChart  allows to develop single ready-to-use graphical widgets (such as histograms, pie graphs, bar graphs, area graphs, scatter diagrams, line graphs, bubble graphs, dispersion graphs) and interactive ones (temporal sliders, add/delete series), to be used separately, by choosing the properties to be included into the reports for a richer data view.



Data Mining and ETL : Tools like WEKA, QBE engine  and Talend  allows advanced data analysis, thanks to Data Mining processes aiming to find out hidden information patterns among a great amount of data.

Combining these tools together Banks can built dashboards, interactive cockpits,  KPI, Adhoc Reporting


There are many open source BI framework solution available

Open source free products

·         Eclipse BIRT Project
·         RapidMiner
·         SpagoBI
·         R
·         KNIME
·         TACTIC

Open source commercial products

·         Jaspersoft: Reporting, Dashboards, Data Analysis, and Data Integration
·         Palo (OLAP database): OLAP Server, Worksheet Server and ETL Server
·         Pentaho: Reporting, analysis, dashboard, data mining and workflow capabilities
·         TACTIC: Reporting, analysis, dashboard, data mining and integration, workflow capabilities

Proprietary free products

·         InetSoft
·         MicroStrategy
·         MicroStrategy Reporting Suite
·         QlikTech
·         QlikView

 

And Large range of  Proprietary products

·         ActiveReports & ActiveReports Server
·         Actuate Corporation
·         Advanced Business Solutions
·         ApeSoft
·         Birst
·         BOARD
·         Business Analyze
·         Cloud9 Analytics
·         ComArch
·         Data Applied
·         Decision Support Panel
·         Dimensional Insight
·         Grapheur, implementing the reactive business intelligence (RBI) approach[2]
·         IBM Cognos
·         IDV Solutions Visual Fusion
·         InetSoft
·         Informatica
·         Information Builders
·         InfoZoom
·         Intellicus
·         Izenda
·         Jackbe
·         Jolicharts
·         JReport (from Jinfonet Software)
·         Klipfolio Dashboard
·         Lavastorm
·         LIONsolver
·         List & Label
·         LogiXML
·         Microsoft
·         SQL Server Reporting Services
·         SQL Server Analysis Services
·         PerformancePoint Server 2007
·         Proclarity
·         Microsoft BI
·         MicroStrategy
·         myDIALS
·         Oracle
·         Hyperion Solutions Corporation
·         Panorama Software
·         Pentaho
·         Pervasive DataRush
·         PRELYTIS
·         ProcessEdge
·         QlikView
·         Business Objects
·         OutlookSoft
·         SAS
·         Siebel Systems
·         Spotfire (now Tibco)
·         Super Office Analyze
·         Sybase IQ
·         Tableau Software
·         TARGIT Business Intelligence
·         Teradata
·         Thomson Data Analyzer
·         XLCubed
·         Zoho Reports (as part of the Zoho Office Suite

SpagoBI is the only entirely Open Source Business Intelligence suite. It covers all the analytical areas of Business Intelligence projects, with innovative themes and engines. SpagoBI offers a wide range of analytical tools, as follows.
SpagoBI analytical engines full list, by Module and Analytical Area.


REPORTING
4 engines: JasperReport, BIRT, Accessible report, BO
SpagoBI allows to realize structured reports, using structured information views (e.g. lists, tables, crosstabs, reports) and to export them using several formats (HTML, PDF, XLS, XML, TXT, CSV, RTF).
·         View example

MULTIDIMENSIONALANALYSIS(OLAP)
3 engines: Jpivot/Mondrian, JPalo/Mondrian, JPXMLA
SpagoBI allows the multidimensional analysis through flexible and user-friendly OLAP engines. Users can monitor the data on different detail levels and from different perspectives, through drill-down, drill-across, slice-and-dice, drill-through processes.
·         View example

CHARTS
3 engines: JFreeChart, HChart, ExtChart
SpagoBI offers a specific graphics engine, based on JFreeChart, which allows to develop single ready-to-use graphical widgets (such as histograms, pie graphs, bar graphs, area graphs, scatter diagrams, line graphs, bubble graphs, dispersion graphs) and interactive ones (temporal sliders, add/delete series), to be used separately, by choosing the properties to be included into the reports for a richer data view.
·         View example

KPI
1 engine: KPI
SpagoBI offers all the necessary tools to create, manage, view and browse KPI hierarchical models, through different methods, calculation rules, thresholds and alarm rules.
·         View example

INTERACTIVE COCKPITS
2 engines: composed document, in-memory (1)
SpagoBI allows users to aggregate several documents into a single view and to interact with them in an dynamic and intuitive way.
·         View example

AD-HOC REPORTING
1 engine: Worksheet
SpagoBI allows end-users to freely create their own multi-sheet reports, including simple tables, cross-tables and different chart types in the document layout.
·         View example n.1
·         View example n.2
·         View example n.3
·         View example n.4

LOCATION INTELLIGENCE
2 engines: GEO, GIS
SpagoBI offers two geographical engines allowing to set run-time connections between geographical data and business data stored in the data warehouse:
·         a GEO engine, which uses a static catalogue in order to display data, allowing users to dynamically re-aggregate the information, according to geographical hierarchies (ex. nation, country, city). This engine can also be used out of the geographical "context" in the strict sense of the word: the indicators distribution can be displayed through any graphical structure (ex. process flow charts, topological schemes of hardware infrastructures)
·         a GIS engine, which interacts with real spatial systems, according to the WFS/WMS scheme.

FREE INQUIRY (Driven Data Selection)
2 engines: QbE, Smart Filter
SpagoBI offers a QbE (Query by Example) engine, which is suitable for those cases in which the free inquiry of data and the extraction of data are more important than their graphical structure and structural layout. Users can build queries through an entirely graphical and web-based interface. They can also execute queries, check the results, export them, save them for future use and generate reporting templates.
·         View example

DATA MINING
2 engines: Weka, R (1)
SpagoBI allows advanced data analysis, thanks to Data Mining processes aiming to find out hidden information patterns among a great amount of data.
·         View example

REAL TIME DASHBOARDS AND CONSOLE
2 engines: Dash, Console
SpagoBI offers a specific engine allowing to produce real-time monitoring consoles, to be used in Business, applicative or BAM processes.

·         View example

COLLABORATION
1 engine: Analytical Dossier
SpagoBI offers a specific engine allowing to automatically create organized report dossiers, enriched with the attending notes and information, posted by users to comment data. A collaborative workflow manages the dossier components to be sent to users in order to receive their notes.

·         View example

OFFICE AUTOMATION
1 engine: Office
SpagoBI offers a specific engine for the publication of personal documents in BI environments, realized through common Office tools (Open Office or MS Office).

ETL
1 engine: Talend
SpagoBI allows to load data into the data warehouse and manage it. SpagoBI ETL engine integrates the open source product TOS (Talend Open Studio).
·         View example

MOBILE
4 engines: Table, Chart, Cockpit, KPI (1)
SpagoBI offers specialized components based on the common touch-screen interaction paradigm, to efficiently combine BI with mobility.
·         View example n.1
·         View example n.2
·         Watch the video

EXTERNAL PROCESSES
1 engine: CommonJ
SpagoBI offers a component that allows to manage customized processes, running in the background and/or starting at a scheduled time.
·         View example

MASTER DATA MANAGEMENT
1 engine: Registry (QbE)
SpagoBI offers a component that allows users to take advantage of write-back functionalities on the database. It allows them to modify the table data through a highly intuitive user interface, whose behavior can be set by means of simple configuration parameters, using pre-defined models available within the platform.
·         View example