We are starting a new blog series that focuses on working with transactions in Spring Cloud Stream Kafka applications. This blog series covers many low-level details of writing transactional applications with Spring Cloud Stream and Apache Kafka. By the end of this blog series, we hope to give you enough information about writing transactional Spring Cloud Stream Kafka applications for various business use cases. Basic Building Blocks The foundational support for transactions in Spring Cloud Stream Kafka applications primarily comes from Apache Kafka itself and the Spring for Apache Kafka…

This blog gives an update on the Schema Registry support that is part of Spring Cloud Stream version 4.0.x. Many enterprises use a schema registry for schema evolution use cases, such as the Confluent Schema Registry. Starting with version 1.1.x of Spring Cloud Stream until 3.0.0, we provided a schema registry server and AVRO-based schema registry client converters that can reconcile the schema from the schema registry server. Spring Cloud Stream version 3.0.0 moved the Schema Registry components into a top-level spring-cloud project, and Spring Cloud Stream included those in its BOM for the…

We are happy to announce that we are incubating a new experimental Spring project for Apache Pulsar. This project aims to provide Spring-friendly APIs, building blocks, and programming models for writing Java applications that interact with Apache Pulsar. Apache Pulsar is a popular messaging system with a growing ecosystem of developers in the enterprise messaging and streaming space. Here are some main features and advantages of using Apache Pulsar for messaging-based software applications: Apache Pulsar provides both the traditional queuing semantics of RabbitMQ, ActiveMQ, and others and the…

In this blog, we are taking a deeper look at writing a Spring Cloud Stream producer with Apache Kafka and how it handles native partitions in Kafka. Spring Cloud Stream has a middleware agnostic concept of partitions. Whenever possible, Spring Cloud Stream leverages the native partitioning capabilities of the middleware if it has such capabilities as in the case of Apache Kafka. This blog looks at how a Spring Cloud Stream developer handles partitions when writing a producer application that publishes data to Kafka. In a subsequent article, we will look at how consumers handle partitions in a…

We are glad to announce the GA release of the newly redesigned Spring Cloud Stream applications - 2020.0.0. We would like to use this release announcement as an opportunity to wrap up the blog series that we started in the summer. Therefore, consider this as part 15 of the blog series. In this blog, we are going to give a rundown of all the previous episodes in the series, but first, let us go through some release details. Release Overview 2020.0.0 GA release contains the completely revamped functional foundation for the event-streaming applications. The old structure was based on an app…

This article is part of a blog series that explores the newly redesigned Spring Cloud Stream applications based on Java Functions. In this post, we will look at the Elasticsearch sink that allows us to index records in Elasticsearch, and its corresponding Consumer function. Here are all the previous parts of this blog series. Introducing Function Based Streaming Applications Function Composition with Streaming Applications How to Build a Supplier and Source Application How to Build a Consumer and Sink Application Build and Run a Simple Stream Application Case Study: HTTP Request Function and…

This article is part of a blog series that explores the newly redesigned Spring Cloud Stream applications based on Java Functions. In this episode, we are exploring the JDBC supplier and the source based on Spring Cloud Stream. We will see how we can export data from a relational database and dump it into a file using a File Consumer and the corresponding Spring Cloud Stream File sink. We will look at a few different ways in which we can run JDBC Source and send the data to a file. Here are all the previous parts of this blog series. Introducing Function Based Streaming Applications Function…

This article is part of a blog series that explores the newly redesigned Spring Cloud Stream applications based on Java Functions. In this episode, we are taking a deeper look into the file supplier and its Spring Cloud Stream file source counterpart. We will also see a MongoDB consumer and its corresponding Spring Cloud Stream sink. Finally, we will demonstrate how the File source and the MongoDB sink can be orchestrated together on Spring Cloud Data Flow as a pipeline. Here are all the previous parts of this blog series. Introducing Function Based Streaming Applications Function Composition…

This is part 4 of the blog series in which we are introducing java functions for Spring Cloud Stream applications. Other parts in the series. Part 1 - General Introduction Part 2 - Function Composition Part 3 - Supplier function and Source application In the last blog in this series, we saw how we can use a java.util.function.Supplier to generate a Spring Cloud Stream source. In this new edition, we will see how a consuming function can be developed and tested using java.util.function.Consumer and java.util.function.Function. Later on, we will briefly explain the generation of a Spring Cloud…

This is part 3 of the blog series in which we are introducing java functions for Spring Cloud Stream applications. Other parts in the series. Part 1 - General Introduction Part 2 - Function Composition In the last two blogs in this series, we provided a general introduction to this new initiative of migrating all the existing Spring Cloud Stream App Starters to functions and the various ways in which we can compose them. In this blog, we continue the series, showing how these functions are developed, tested, and used to generate Spring Cloud Stream applications. In particular, here we are…