Future from your data
Our IT consultants
understand your strategy

Talk to us

Request further information and demos now, free of charge and without obligation!
Write us about your project.

Variable targeted information (contact for current topics, events, demo, job profiles) / contact form general as well as for trade fair and demo

* mandatory fields

Ihre Nachricht wurde erfolgreich übermittelt. In Kürze wird sich ein Mitarbeiter via E-Mail bei Ihnen melden.
Ihre Nachricht konnte nicht übermittelt werden. Bitte prüfen Sie Ihre Eingaben.

Business Case: Big Data Management for comparison portal Billiger.de

Hadoop implementation during operation

Project requirements/tasks

The website "billiger.de", a brand of solute GmbH, is one of the largest and most clicked online price comparison portals in Germany. This platform helps consumers to find low-priced offers at online shops. Detailed product information makes it easier to decide between millions of offers from thousands of shops. These offers are updated several times a day, ensuring transparency and an overview of the online market for retailers and consumers. The amount of data to be processed is correspondingly high.

Hadoop is a free framework which, thanks to its interfaces and scalability, can be used flexibly, cost-effectively, efficiently and future-proof, especially with polystructured mass data. Internet giants such as Google, Yahoo and Facebook have long shown that Hadoop technology is currently the best suited to handle such data volumes. 

solute created an appropriate system on a "greenfield site" and quickly realized that external consulting was needed due to the complexity and scope of the project.

© Billiger.de

Price comparison portals such as Billiger.de require a modern infrastructure in order to serve the different channels and to be able to specifically evaluate the diverse data.

This is our contribution

Hadoop specialist KYBEIDOS began integrating the Hadoop system into the existing IT architecture of "billiger.de" after a brief requirements analysis and architecture consulting - and this during ongoing operations. Thanks to the system configuration tool "Puppet" all installations and configurations could be carried out fully automatically. The decisive factors were status monitoring, the detection of failures and the analysis and correction of errors. Furthermore, the configuration was adapted to specific use cases. Penthao DI was used for data enrichment and processing. In addition, Kafka, Flume and HBase, a scalable, simple database for managing very large amounts of data within a Hadoop cluster, was used.

The Hadoop system was then upgraded to the latest version and connected to the Microsoft Windows user management system. Four months after the start of the project, KYBEIDOS had successfully fulfilled all requirements and at the same time provided flexible support for the BI department.

KYBEIDOS implemented our requirements professionally and advised all project partners very competently. I was also impressed by how flexibly and fairly KYBEIDOS reacted to changing circumstances.

Gunter Bach, Director IT solute GmbH │ „billiger.de“

Benefits at a glance

    N
  • Integration of the Hadoop system into the existing IT architecture

  • N
  • Upgrade your Hadoop system to the latest version

  • N
  • Connection of the new Hadoop system to the Windows user administration

  • N
  • Status monitoring, error analysis and correction

  • N
  • Maintenance of ongoing operations

  • N
  • Technical consulting for the IT, business and BI department

The project at a glance

Branche:

  • E-Commerce

  • Comparison portal

  • Medium-sized companies

Project partners:

  • Management

  • Specialist-Department

  • IT-Department

  • BI-Department

Technologies:

  • Hadoop
  • Kafka
  • Puppet
  • HBase
  • Flume
  • Pentaho DI
  • Flume

Bedarf:

  • Integration eines Hadoop-Systems in die vorhandene Infrastruktur

Kybeidos Beitrag:

  • Beratung (techn./fachl.)
  • Tool-Evaluierung
  • Systemüberführung
  • Systemanbindung
  • Upgrade
  • Operating
  • Monitoring
  • Support