Huawei India R&D Center makes major contributions to the Big Data Analytics’ industry initiative

Huawei India R&D Center, the largest overseas R&D center, and a key platform/component development & delivery center of Huawei announced a breakthrough in big data domain through its industry collaboration initiatives. The India R&D center in Bengaluru has contributed to CarbonData project which was formally incubated to Apache Software Foundation by Huawei in June 2016.

Apache CarbonData (incubating) is a fully indexed columnar and Hadoop native data-store for processing heavy analytical workloads and detailed queries on big data. What makes Carbon Data so special is its unified storage format which is designed and developed from the ground up to provide faster response for different user queries. It is columnar and supports Global Dictionary with Deferred Decoding, multi-level indexing and column groups making it unique and fast. It has deep integration with Spark, allowing spark users to readily use it. Most organizations use different data-stores and query engines to meet their use cases of batch Queries (Big scan), Interactive Analysis Queries (Multi-Dimensional OLAP) and Operational Queries (Random Access / Narrow Scan) which results in duplication of data and processing, thereby increasing the cost. This is one of the key challenges that CarbonData intends to solve.

Today CarbonData, within 6 months of its incubation into Apache, has attracted more than 60 active contributors with steadily growing user base and is becoming a unified storage solution for big data analytics.

Huawei India R&D center has identified the need for collaboration and contribution to open source years ahead and has been playing an active role in executing Huawei’s open source strategy. The R&D center has contributed to open source projects in SDN, NFV, Cloud, Big data and analytics domains, and with the support of strong local eco-system is driving innovations in various domains through community collaboration and cooperation.