Zuchos.com

In my previous post I created Publisher for akka-streams that was buffering incoming data and then passing those down the stream. Johannes Rudolph aptly observed that the flow of that solution is the buffer overflow scenario (too many incoming requests may lead to out-of-memory issue).

Thanks for the post! It’s nice to see that people are actually starting to use akka-stream and akka-http. A note: implementing ActorPublisher shouldn’t be necessary in most cases. In this case you built an unbounded buffer in front of a stream which defeats akka-stream/reactive streams back pressure logic. Now if the consumer cannot keep up with reading the data all the unwritten data will start to pile up in memory. Generally, it isn’t possible to switch from a pull-style (akka-stream/reactive-stream) model to a push-style model (actor message tell) somewhere in the processing chain. In cases where you still need to do this (e.g. because you are dealing with a “live” data source) there’s a somewhat safer option: use Source.actorRef which lets you define a limited buffer and makes you choose a strategy what to do when the buffer is full. Johannes Rudolph

First I would like to explain my motivation, this case comes from my pet project. In that project I’m expecting that users around the world will send me the data, so I want to make API as simple as possible (what could be simpler than REST API?). Users are not interested in the result of computation, they are interested in contributing the data. So the system should accept as many data as possible (return status 202 - Accepted to the user - it would mean that we received the data) and then process it with it’s own speed. I rather expect to have many request from different user, than one user will be sending tons of those. The buffer overflow is possible situation here so Johannes was right that it should be tackled. First I took a look on proposed solution Source.actorRef(). The problem with ActorRefSourceActor is that the all available OverflowStrategy values are not notifying sender that the problem occurred and that leads to lost of data. So I couldn’t use that solution.

So I came up with different one, I added bufferSize val to DataPublisher and in receive method I extracted cacheIfPossible() method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

  override def receive: Actor.Receive = {
    case Publish(s) =>
      cacheIfPossible(s)
    case Request(cnt) =>
      publishIfNeeded()
    case Cancel => context.stop(self)
    case _ =>
  }

  private def cacheIfPossible(s: Data) {
    if (queue.length == bufferSize) {
      sender() ! Failure(new BufferOverflow)
    } else {
      queue.enqueue(s)
      sender() ! Success()
      publishIfNeeded()
    }
  }

Recently I started using akka-http and what I was trying to achieve was to receive data from request, send response that the data were received successfully and then process it asynchronously. The other requirement was that the processing flow could be complicated in the future and some parts of it could be faster than other, so I decided to use akka-streams for that. I started with empty akka-http service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

  trait SimpleService {

    implicit val system: ActorSystem
    implicit def executor: ExecutionContextExecutor
    implicit val materializer: FlowMaterializer

    val routes = {
      path("hello") {
        get {
          complete("Hello World!")
        }
      }
    }
  }

  object NaiveGsServer extends App with SimpleService {

    override implicit val system = ActorSystem()
    override implicit val executor = system.dispatcher
    override implicit val materializer = ActorFlowMaterializer()

    val config = ConfigFactory.load()

    Http().bindAndHandle(routes, config.getString("http.host"), config.getInt("http.port"))

  }

I like this time of the year. Summers just begins, it’s nice and warm outside the sun is shining and my favorite community conference takes place. Confitura is very close to my heart. I had a pleasure to participate in the organization in the past. But this year I was on the other side, a speaker. I was a bit curious how community will react on the topic of my speech (“How to be a happy Developer?”)? Would it catch attention and fill the room? It was a “soft” presentation after all.

Not so long time a go, I was on the conference about Agile (AgileByExample - great conference by the way) and one of the sponsors has a win-tablet contest. To win brand new Nexus you had to post on their facebook wall answer for a question:

What does Agile mean to you?

I posted 2 answers but the contest was setup and someone else won my Nexus (just kidding of course). Conference was over but the question was coming back, ringing in my head. What the hell is Agile ? Is it methodology (ugly word), nickname for Scrum, XP or maybe just a buzzword? Other questions emerged: Who could be agile? Developer? Analyst? CEO? Housewife? My confusion was rising… I knew that I had to start from the beginning. What is called Agile by most of the people? For sure one of those:

• TDD
• Scrum
• Kanban

What are the similarities?

TDD gives me flexibility to change my code anytime and I know that it will work as I’m expecting. Scrum (same as Kanban) helps me to deal with changing (or just discovered) customer’s requirements. Main difference between agile approach and its opposite: waterfall (or simply planned end-to-end process) is ability to handle change. So simply agile means being adaptive. Adaptation is very important thing, in fact it’s a matter of live or dead.

It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change. Charls Darvin

So if you don’t want to end up like mammoth or dinosaurs, be adaptive! I think that the reason why nowadays start-ups and small innovative companies are so successful. They are adaptive!

In SML we do code reviews. We do them on daily basis. Actually the point that we are now is a result of long journey that we made. We try different strategies and tools until we went to the place that we are now (but it doesn’t mean that we are going to end up here).<

During this journey we found many risks and traps that are waiting for a newcomer. That’s what this post is all about, traps & misconceptions on code review.

Code control: many organizations uses CR for controlling codebase. Most of them are using pre-commit strategy. In many cases its because those projects are open-source with hundred of commiters. In real life this is quite rare scenario, so if you hired someone it means that you trust him enough to let him commit code to repository. I know that in some organizations there will be temptation to make procedure that will force developers to “review” and “approve” every commit, but it will not guarantee the quality. Moreover people will soon treat code review as “stupid” corporate procedure and will try to hack-it (such changing password every month e.g. people are using passwords like: mypass1, mypass2 etc.).