Aiven talks shop with Paf at second Apache Kafka meetup

In August, Aiven created an Apache Kafka meetup in Helsinki to discuss hot topics surrounding Kafka.

Due to its popularity, we decided to make it an ongoing event and held our second meetup yesterday.

This time, around 30 people attended Lifeline Ventures' office in downtown Helsinki and there were two presenters:
  1. Niklas Nylund of Paf, an operator of slots, lotteries, poker and casino games as well as betting both online and in casinos.
  2. Heikki Nousiainen, our CTO
First up was Nylund to discuss the ins and outs of how his team brought Kafka in to act as a data bus for their real-time analytics needs.

Kafka reduces the spaghetti  

Paf collects Change Data Capture events from databases and sends the events to Kafka, which are then consumed and imported into Kudu for analytic work.

Although Kafka possesses its quirks, Nylund is satisfied with how Paf has been able to use it to streamline their architecture.

Or, as he vividly put it, "Reducing the spaghetti."

Check out his presentation slides here to get a thorough understanding of Paf's integration process from beginning to end.

Kafka Connect simplifies integration

Second up was our CTO to discuss Kafka Connect framework, and using it to transfer data between Kafka and other systems, in this case
PostgreSQL and Elasticsearch.

Using Python code to interact with the services, Nousiainen was able to demonstrate how easy it was to push and pull data between Kafka and the external systems.

With a large number of available connectors from the Kafka community, integrating Kafka with other systems can be quite straightforward.

This in turn allows quick benefits and migration towards real-time stream analytics with Kafka-centric architecture.

Check out Nousiainen's presentation slides to get a better idea of the use cases for Kafka Connect.

Join the next Kafka discussion  

As the transition from a monolithic to microservices architecture continues, the use case for integrating Kafka as a streaming platform will only strengthen. 

This is evidenced by the increase in attendance of our events where many developers, be they users of Kafka or not, are gathering to learn more about what Kafka is, its use cases, and best practices for implementing it.

We are planning another Aiven Kafka meetup for December/January timeframe, so join the Helsinki Apache Kafka Meetup group to get the details when we finalize the plans and we'll see you soon!


Aiven is the first to offer PostgreSQL 10

Get Aiven PostgreSQL 10
We've got great news: PostgreSQL 10 is now available at Aiven on all major clouds!

That means that you can now access it on AWS, Google, Azure, UpCloud, and DigitalOcean clouds. Worldwide.

As with every PostgreSQL release, many of your older queries will simply run faster because of the many performance enhancements. But, why is PostgreSQL 10 significant?

As the 28th major update of the past 30 years, its primary focus is on improving the distribution of massive amounts of data across many nodes...let's look at some specifics.

PostgreSQL 10: the specifics 

Improved support for parallel queries 

Now, many more scan types are supported and can benefit from parallelization.

Depending on your query, newly added scan types such as parallel index scan and bitmap heap scan can speed it up immensely.

Also, merge joins are now a supported parallel join type in addition to other join types already supported in the previous release, such as hash joins and nested loop joins.

    Declarative partitioning support. 

    While you could create partitioning schemes by directly using constraints, inheritance and triggers in past versions of PostgreSQL...

    ...you can now use simple definitions to create your partitioning setup with PostgreSQL 10

    Even better, the performance of the new partitioning code is vastly improved over older methods.

      Hash indexes

      PostgreSQL 10 brings crash-safe hash index support that also performs far better than before. 

      Now, you can consider using hash indexes when your queries just need to check for equivalence to increase performance.

        Native logical replication 

        PostgreSQL 10 now brings proper support for logical replication in PostgreSQL itself. 

        Logical replication allows replication between different PostgreSQL versions, finally allowing for zero downtime upgrades to future versions. 

        You can also migrate data to and from environments where you don't have access to streaming replication.

        Easily test PostgreSQL 10 with Aiven today  

        With Aiven, you can create a full copy of the data within your existing PostgreSQL service as a new separate PostgreSQL 10 service.

        By forking, you can keep your existing PostgreSQL services as-is while testing the latest version for compatibility with your applications.

        And don't worry, it won't negatively affect the performance of your source service; it just provides an easy and efficient way to test PostgreSQL 10. So, let's get started.

        Team Aiven

        P.S. For a full list of features please see the full PostgreSQL 10 release notes.


        Stanford SSI uses Aiven InfluxDB in high-altitude

        Aiven goes to near space with InfluxDB

        A new day in space development 

        Historically, space development has remained the sole purview of government agencies due to its astronomical cost: that has changed. 

        Today, new technologies are reducing those costs, allowing smaller groups with big ambitions to take small steps and giant leaps in pushing space development forward. 

        One such group is Stanford SSI, a project-based student group that covers everything from high-altitude balloon platforms, to cube satellites, all the way through to rockets. 

        In fact, they recently broke the world record for the longest duration flight by a latex balloon with the launch of SSI-52...a pretty big achievement for a group only established in 2013. 

        What does Aiven have to do with balloons? 

        These aren’t your everyday balloons, but sophisticated platforms carrying complex scientific payloads into near-space and over tremendous distances: the stakes are a little higher. 

        Stanford SSI uses its ValBal balloon platform to reach altitudes as high as 120,000 feet while testing cutting-edge electronics and mechanics. 

        Their tests produce a massive amount of data, data that needs to be sent back. Kai Marshland, their Operations Lead, explains it best, 

        “...we want to store a highly variable set of data, analyze it over time, and manipulate it with minimal latency...Aerospace demands the utmost in quality, and that’s exactly what Aiven provides.” 

        In short, Aiven’s capabilities provide an ideal fit. For our part, we think it’s pretty cool to test our technology in demanding, high-altitude research flights. 

        Aiven tests InfluxDB at the edge of space 

        For latest launch on September 30, 2017, SSI-59, Stanford SSI used our InfluxDB service for their Database as a Service needs. 

        The goal for this launch? SSI-59 will use the same ValBal platform to test lighter and more efficient mechanics and avionics that should increase the platform’s endurance. 

        But most importantly, it’s the communications system that they are most excited to test, which Kai describes as “Revolutionary.” Here’s why, 

        "Having high-bandwidth communications means that we no longer have to worry about recovering the payload over the vast areas ValBal can fly over." 

        For instance, it will allow them to fly a radar glaciology payload over Greenland to measure the thickness of its ice sheets and transmit the data it collects almost instantaneously back to where they launched the payload from...no more worrying about recovering the platform.

        We operate in the cloud, but reach for the stars 

        Just one of many, SSI-59 is part of a long-term aim to redefine the high-altitude balloon research world, one that will provide a better understanding of the planet we live on. 

        This is the first time that Stanford SSI will be using our technology for their data needs and will provide an excellent use case for demonstrating the capabilities of our technology. 

        But, the idea that our technology can be applied to space development is most thrilling. After all, Aiven may be a database cloud service provider, but we don’t mind flying above them.