#apache project

[techfindin]

Synapse: Another Apache ESB

Apache Synapse

Apache Synapse is a lightweight and high-performance Enterprise Service Bus (ESB). Powered by a fast and asynchronous mediation engine, Apache Synapse provides exceptional support for XML, Web Services and REST. In addition to XML and SOAP, Apache Synapse supports several other content interchange formats, such as plain text, binary, Hessian and JSON. The wide range of transport adapters available for Synapse, enables it to communicate over many application and transport layer protocols. As of now, Apache Synapse supports HTTP/S, Mail (POP3, IMAP, SMTP), JMS, TCP, UDP, VFS, SMS, XMPP and FIX.

Apache Synapse is free and open source software distributed under the Apache Software License 2.0. The latest version of Synaspe is v2.1. This release comes with a horde of new features, bug fixes, performance and stability improvements.

Apache Synapse, Synapse, Apache, the Apache feather logo, and the Apache Synapse project logo are trademarks of The Apache Software Foundation

New in Version 2.1

    Support for creating and managing REST APIs with URL patterns and URI templates

    Define sequences and endpoints as templates and reuse seamlessly across the whole configuration

    Payload factory mediator for constructing custom XML payloads

    Recipient list endpoint

    HTTPD like access logging capability in the NHTTP transport

    Message stores and message processors for implementing complex integration patterns and advanced messaging scenarios

    Receiving sequence support in the send mediator to make service chaining easier

    Conditional router mediator for implementing complex routing scenarios with many conditions and rules

Key Features

    Proxy services - facilitating transport, interface (WSDL/Schema/Policy), message format (SOAP 1.1/1.2, POX/REST, Text, Binary), QoS (WS-Addressing/ WS-Security/WS-RM) and optimization switching (MTOM/SwA)

    Non-blocking HTTP/S transports for fast HTTP interactions and support for thousands of concurrent connections

    VFS transport for file manipulation and interaction with FTP, SFTP, CIFS and WEBDAV

    JMS support for binary, plain text, XML and SOAP payloads

    Mail transport with extensive support for POP3, IMAP and SMTP

    Support for industry driven Financial Information eXchange (FIX) protocol

    Built-in Registry/Repository, facilitating dynamic reloading of the configuration and associated resources (e.g. XSLTs, XSD, JS, ..)

    Built-in support for scheduling tasks using the Quartz scheduler

    Load-balancing (with or without sticky sessions) and fail-over routing

    Support for many Web Services standards including WS-Addressing, WS-Security and WS-Reliable Messaging

    Policy based message throttling and caching (with special support for clustered environments)

    Message splitting and aggregation

    Database lookup and update support with database connection pooling

    Fine grained statistics collection over sequences, endpoints and proxy services

    JMX monitoring and management

    Easily extendable with Java, Spring, or BSF Scripting languages (Javascript, Ruby, Groovy, etc.)

[techfindin]

Kafka: Linking up Messages

Kafka[http://incubator.apache.org/kafka/]

Kafka is a messaging system that was originally developed at LinkedIn to serve as the foundation for LinkedIn's activity stream and operational data processing pipeline. It is now used at a variety of different companies for various data pipeline and messaging uses.

Activity stream data is a normal part of any website for reporting on usage of the site. Activity data is things like page views, information about what content was shown, searches, etc. This kind of thing is usually handled by logging the activity out to some kind of file and then periodically aggregating these files for analysis. Operational data is data about the performance of servers (CPU, IO usage, request times, service logs, etc) and a variety of different approaches to aggregating operational data are used.

In recent years, activity and operational data has become a critical part of the production features of websites, and a slightly more sophisticated set of infrastructure is needed.

[techfindin]

Flume: Aggregating Huge Data

Flume[https://cwiki.apache.org/FLUME/]

Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. The system is centrally managed and allows for intelligent dynamic management. It uses a simple extensible data model that allows for online analytics application.

Flume is open-sourced under the Apache Software Foundation License v2.0.

[techfindin]

Gora: ORM for Big Data

Gora[http://gora.apache.org/]

What is Apache Gora?

The Apache Gora open source framework provides an in-memory data model and persistence for big data. Gora supports persisting to column stores, key value stores, document stores and RDBMSs, and analyzing the data with extensive Apache Hadoop MapReduce support.

Why Apache Gora?

Although there are various excellent ORM frameworks for relational databases, data modeling in NoSQL data stores differ profoundly from their relational cousins. Moreover, data-model agnostic frameworks such as JDO are not sufficient for use cases, where one needs to use the full power of the data models in column stores. Gora fills this gap by giving the user an easy-to-use in-memory data model and persistence for big data framework with data store specific mappings and built in Apache Hadoop support.

The overall goal for Gora is to become the standard data representation and persistence framework for big data. The roadmap of Gora can be grouped as follows.

Data Persistence : Persisting objects to Column stores such as HBase, Cassandra, Hypertable; key-value stores such as Voldermort, Redis, etc; SQL databases, such as MySQL, HSQLDB, flat files in local file system of Hadoop HDFS.

Data Access : An easy to use Java-friendly common API for accessing the data regardless of its location.

Indexing : Persisting objects to Lucene and Solr indexes, accessing/querying the data with Gora API.

Analysis : Accesing the data and making analysis through adapters for Apache Pig, Apache Hive and Cascading

MapReduce support : Out-of-the-box and extensive MapReduce (Apache Hadoop) support for data in the data store.