On Scala

Readings
- Actor model (Wikipedia)
- Understanding actor concurrency (Feb 24, 2009)
  - key principles of actor model
    - No shared state
    - Lightweight processes
    - Asynchronous message-passing
    - Mailboxes to buffer incoming messages
    - Mailbox processing with pattern matching
- Actor Path
  - An actor path consists of an anchor, which identifies the actor system, followed by the concatenation of the path elements, from root guardian to the designated actor; the path elements are the names of the traversed actors and are separated by slashes.
- Getting Started Tutorial (Java): First Chapter (for Akka 2.0.2)
- Building a scalable and highly available reactive applications with Akka! Load balancing revisited(July 15, 2014)
- Why We Picked AKKA Cluster as our Microservice Framework
- Discovering message flows in actor systems with the Spider Pattern
- Blocking Needs Careful Management
  - The non-exhaustive list of adequate solutions to the “blocking problem” includes the following suggestions:
    - Do the blocking call within an actor (or a set of actors managed by a router [Java, Scala]), making sure to configure a thread pool which is either dedicated for this purpose or sufficiently sized.
    - Do the blocking call within a Future, ensuring an upper bound on the number of such calls at any point in time (submitting an unbounded number of tasks of this nature will exhaust your memory or thread limits).
    - Do the blocking call within a Future, providing a thread pool with an upper limit on the number of threads which is appropriate for the hardware on which the application runs.
    - Dedicate a single thread to manage a set of blocking resources (e.g. a NIO selector driving multiple channels) and dispatch events as they occur as actor messages.
- Akka Cluster Samples with Java
- Akka Camel Samples with Java (April 14, 2016)
- Akka Java Spring
- Akka cluster: how to disable ClusterHeartbeat log (Apr 28 '15)
- Akka Utils: get a single ActorRef for a path
- Routing
  - AkkaRouterExample
  - Core concepts for routing
    - akka.routing.Router : default and only router implementation, not inheritable and immutable
    - akka.routing.RouterConfig : router factory
    - akka.routing.RoutingLogic : routing logic, should be thread-safe
    - akka.routing.Routee : destination for messages routed via a router
  - built-in Akka routers and alias

   akka.actor.router.type-mapping {
      from-code = "akka.routing.NoRouter"
      round-robin-pool = "akka.routing.RoundRobinPool"
      round-robin-group = "akka.routing.RoundRobinGroup"
      random-pool = "akka.routing.RandomPool"
      random-group = "akka.routing.RandomGroup"
      balancing-pool = "akka.routing.BalancingPool"
      smallest-mailbox-pool = "akka.routing.SmallestMailboxPool"
      broadcast-pool = "akka.routing.BroadcastPool"
      broadcast-group = "akka.routing.BroadcastGroup"
      scatter-gather-pool = "akka.routing.ScatterGatherFirstCompletedPool"
      scatter-gather-group = "akka.routing.ScatterGatherFirstCompletedGroup"
      tail-chopping-pool = "akka.routing.TailChoppingPool"
      tail-chopping-group = "akka.routing.TailChoppingGroup"
      consistent-hashing-pool = "akka.routing.ConsistentHashingPool"
      consistent-hashing-group = "akka.routing.ConsistentHashingGroup"
    }

- Spring
  - Typesafe Activator template for Akka and Spring in Java

Spark

https://spark.apache.org/
Desc. : a fast and general engine for large-scale data processing.
License :
Written in : Scala
Maven artifacts : org.apache » spark

References
- Official documentation
- Scala API
  - Spark 2.0.0 Scala API
  - Spark 1.6.2 Scala API
    - org.apache.spark.SparkConf
      - Configuration for a Spark application. Used to set various Spark parameters as key-value pairs.
    - org.apache.spark.SparkContext
      - Main entry point for Spark functionality. A SparkContext represents the connection to a Spark cluster, and can be used to create RDDs, accumulators and broadcast variables on that cluster.
    - org.apache.spark.rdd.RDD
    - org.apache.spark.streaming.dstream.DStream
    - org.apache.spark.streaming.dstream.PairDStreamFunctions
- Java API
  - Spark 1.6.2 Java API
  - Spark 1.2.0 Java API
- Databricks Spark Reference Applications

Books
- Mastering Apache Spark
- Faster Stateful Stream Processing in Apache Spark’s Streaming * ebruary 1, 2016)

Readings

Performance
- Controlling Parallelism in Spark
- How-to: Tune Your Apache Spark Jobs (Part 1) (March 9, 2015)
- How-to: Tune Your Apache Spark Jobs (Part 2) (March 30, 2015)

Spark Streaming
- Spark Streaming Example – How to Stream from Slack
- Faster Stateful Stream Processing in Apache Spark’s Streaming (February 1, 2016)
- How-to: Do Near-Real Time Sessionization with Spark Streaming and Apache Hadoop (November 3, 2014)

Spark Testing
- Spark and Spark Streaming Unit Testing (Mar 1st, 2015)
- Spark Streaming Testing with Scala Example
- Testing Spark: Best Practices (2014/06)

Spark on Windows
- Running Spark on Windows
- winutils
  - Windows binaries for Hadoop versions
- Getting started with Apache Spark and Scala on Windows (2 March 2016)

misc
- Java vs Scala vs python for big data apache spark project(2016/2/10)
- Scala vs. Python for Apache Spark (12 Feb 2016)
  - Scala is 10 times faster than Python.
  - Scala's arcane syntax makes it difficult to master.

Integration

Kafka

http://kafka.apache.org/
Desc. : publish-subscribe messaging rethought as a distributed commit log.
License : Apache License
Written in : Scala for core, Java for clients and tools
Sources : https://github.com/apache/kafka
Maven artifacts : org.apache.kafka

References
- Official documentation
  - Kafka 0.10.0 Documentation
    - Broker Configs
  - Kafka 0.9.0 Documentation
- API
  - Kafka 0.10.0.1 Java API
  - Kafka 0.9.0.1 Scala API

Readings

Associates

CSharpClient for Kafka

https://github.com/Microsoft/CSharpClient-for-Kafka
Desc. : .Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes.
Sources :

Spring for Apache Kafka

https://projects.spring.io/spring-kafka/
Desc. : Provides Familiar Spring Abstractions for Apache Kafka
Sources : https://github.com/spring-projects/spring-kafka

Kafka Manager

https://github.com/yahoo/kafka-manager
Desc. : A tool for managing Apache Kafka.

Readings
- Install and Evaluation of Yahoo's Kafka Manager (3/29/2015)

Testing

ScalaTest

http://www.scalatest.org/
Desc. : the most flexible and most popular testing tool in the Scala ecosystem
License :
Maven artifacts : org.scalatest

Readings
- ScalaTest 2.2.6 API
  - org.scalatest.FunSuite class

Libraries

JSON

Json4s

http://json4s.org/
Desc. : A single AST to be used by other scala json libraries
License :
Sources : https://github.com/json4s/json4s
Readings

NoSQL Client

Casbah=

https://github.com/mongodb/casbah
Desc. : an interface for MongoDB designed to provide more flexible access from both Java and Scala.
License : Apache License, Version 2.0

Tools

sbt

http://www.scala-sbt.org/
Desc. : interactive build tool
License :

Readings
- sbt 0.13 Reference Manual
- sbt 0.13 API

Scala.js

http://www.scala-js.org/
Desc. : the Scala to JavaScript compiler
License : Scala License
Sources : https://github.com/scala-js/scala-js

On Scala

Contents

Language

Grammar

Collection

Concurrency

Logging

External Process

Functional Programming

misc

Frameworks

IoC

Spring Scala

MVC

Play

Persistency

MyBatis Scala

Networking

Finagle

Distributed

Akka

Spark

Readings

Integration

Kafka

Associates

CSharpClient for Kafka

Spring for Apache Kafka

Kafka Manager

Testing

ScalaTest

Libraries

JSON

Json4s

NoSQL Client

Casbah=

Tools

sbt

Scala.js

Fan Feed