On Scala

Language

• Scala API Docs
  • Scala 2.12.1 Library API
    • scala.Predef object API
    • scala.collection.immutable.StringOps class API
  • Scala 2.11.8 Library API

• Sources
  • scala/scala on GitHub
  • scala.package.scala source
  • scala.App source

• Official Scala documentation
  • Tutorials
  • Scala Tools and Utilities
  • Scala Style Guide

• Scala 2.11 Language Specification

• Scala Cookbook
• Scala School!
• Effective Scala

• The Rise and Fall of Scala (Oct. 06, 16)

Grammar

• Scala _ (underscore) magic (29 Mar 2010)
• Quick Explanation of Scala’s (_+_) Syntax (9 Jan 2008)

Collection

• Scala’s Collections Library
• Arrays
• Convert from scala.collection.Seq<String> to java.util.List<String> in Java code (Jul 19 '13)
• Five ways to create a Scala List (October 27 2016)
  • Lisp style(1 :: 2 :: 3 :: Nil), List(1,2,3), List.range(1, 3), ...

Concurrency

• Parallel programming in Go and Scala : A performance comparison

Logging

• error with slf4j: ambiguous reference to overloaded definition (21 Dec 2012)
  • logger.debug("hello {} / {} ", "Hello", "World":Any)

External Process

• scala.sys.process package
• scala.sys.process.ProcessBuilder trait
• How to download URL contents to a String or file (June 3 2016)
  • new URL("http://google.com") #> new File("Output.html") !!
• How to execute (exec) external system commands in Scala (June 3 2016)

Functional Programming

• A Beginner-Friendly Tour Through Functional Programming in Scala (Dec. 13, 16)
• Higher Order Functions

misc

• Java + Scala
• Start/Deploy Apache Spark application programmatically using spark launcher (June 26, 2015)
• Null, null, Nil, Nothing, None, and Unit in Scala (July 12, 2009)

Frameworks

IoC

Spring Scala

• http://hub.darcs.net/psnively/spring-scala
• Desc. : make it easier to use the Spring framework in Scala.
• License :
• Sources

MVC

Play

• https://www.playframework.com/
• Desc. : The High Velocity Web Framework For Java and Scala
• License

Persistency

MyBatis Scala

• http://www.mybatis.org/scala/
• Desc. :
• License

Networking

Finagle

• http://twitter.github.io/finagle/
• Desc. : an extensible RPC system for the JVM, used to construct high-concurrency servers.
• License :
• Written in : Scala
• Sources : https://github.com/twitter/finagle
• Readings
  • Finagle User's guide
  • Finagle Scala API

Distributed

• Akka vs Storm (25 June, 2013)

Akka

• http://akka.io/
• Desc. : a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM.
• License : Apache 2 License
• Written in :
• Sources : https://github.com/akka/akka
• References
  • Akka 2.4.4
    • Java Documentation
      • Akka HTTP
    • Java API
      • akka.routing.RouterActor class
    • Scala 2.11.7 API
  • Akka 2.3.15
    • Java Documentation
      • Routing
      • Cluster Client
    • Java API
      • akka.actor.ActorSystem class
        • void registerOnTermination(java.lang.Runnable code) method
      • akka.cluster.Cluster class
      • akka.routing.RouterConfig class
      • akka.japi.Util class
    • Akka Actor reference config file
    • Akka Cluster reference config file
  • Typesafe Config Library API
    • static com.typesafe.config.ConfigFactory.load() method
    • static com.typesafe.config.ConfigFactory.defaultApplication method
      • application.conf, application.json, application.properties, config.file, config.resource, and config.url

• Books
  • Applied Akka Patterns, 2016, Wade Waldron, Michael Nash

• Readings
  • Actor model (Wikipedia)
  • Understanding actor concurrency (Feb 24, 2009)
    • key principles of actor model
      • No shared state
      • Lightweight processes
      • Asynchronous message-passing
      • Mailboxes to buffer incoming messages
      • Mailbox processing with pattern matching
  • Actor Path
    • An actor path consists of an anchor, which identifies the actor system, followed by the concatenation of the path elements, from root guardian to the designated actor; the path elements are the names of the traversed actors and are separated by slashes.
  • Getting Started Tutorial (Java): First Chapter (for Akka 2.0.2)
  • Building a scalable and highly available reactive applications with Akka! Load balancing revisited(July 15, 2014)
  • Why We Picked AKKA Cluster as our Microservice Framework
  • Discovering message flows in actor systems with the Spider Pattern
  • Blocking Needs Careful Management
    • The non-exhaustive list of adequate solutions to the “blocking problem” includes the following suggestions:
      • Do the blocking call within an actor (or a set of actors managed by a router [Java, Scala]), making sure to configure a thread pool which is either dedicated for this purpose or sufficiently sized.
      • Do the blocking call within a Future, ensuring an upper bound on the number of such calls at any point in time (submitting an unbounded number of tasks of this nature will exhaust your memory or thread limits).
      • Do the blocking call within a Future, providing a thread pool with an upper limit on the number of threads which is appropriate for the hardware on which the application runs.
      • Dedicate a single thread to manage a set of blocking resources (e.g. a NIO selector driving multiple channels) and dispatch events as they occur as actor messages.
  • Akka Cluster Samples with Java
  • Akka Camel Samples with Java (April 14, 2016)
  • Akka Java Spring
  • Akka cluster: how to disable ClusterHeartbeat log (Apr 28 '15)
  • Akka Utils: get a single ActorRef for a path
  • Routing
    • AkkaRouterExample
    • Core concepts for routing
      • akka.routing.Router : default and only router implementation, not inheritable and immutable
      • akka.routing.RouterConfig : router factory
      • akka.routing.RoutingLogic : routing logic, should be thread-safe
      • akka.routing.Routee : destination for messages routed via a router
    • built-in Akka routers and alias

   akka.actor.router.type-mapping {
      from-code = "akka.routing.NoRouter"
      round-robin-pool = "akka.routing.RoundRobinPool"
      round-robin-group = "akka.routing.RoundRobinGroup"
      random-pool = "akka.routing.RandomPool"
      random-group = "akka.routing.RandomGroup"
      balancing-pool = "akka.routing.BalancingPool"
      smallest-mailbox-pool = "akka.routing.SmallestMailboxPool"
      broadcast-pool = "akka.routing.BroadcastPool"
      broadcast-group = "akka.routing.BroadcastGroup"
      scatter-gather-pool = "akka.routing.ScatterGatherFirstCompletedPool"
      scatter-gather-group = "akka.routing.ScatterGatherFirstCompletedGroup"
      tail-chopping-pool = "akka.routing.TailChoppingPool"
      tail-chopping-group = "akka.routing.TailChoppingGroup"
      consistent-hashing-pool = "akka.routing.ConsistentHashingPool"
      consistent-hashing-group = "akka.routing.ConsistentHashingGroup"
    }

• • Spring
    • Typesafe Activator template for Akka and Spring in Java

Spark

• https://spark.apache.org/
• Desc. : a fast and general engine for large-scale data processing.
• License :
• Written in : Scala
• Maven artifacts : org.apache » spark

• References
  • Official documentation
    • Spark 2.0.1 Documentation
      • Spark Configuration Properties
      • Spark Standalone Mode
      • spark-submit
    • Spark 1.6.2 Documentation
      • Spark Configuration Properties
      • Spark Standalone Mode
    • Spark 1.2.0 Documentation
  • Scala API
    • Spark 2.0.0 Scala API
    • Spark 1.6.2 Scala API
      • org.apache.spark.SparkConf
        • Configuration for a Spark application. Used to set various Spark parameters as key-value pairs.
      • org.apache.spark.SparkContext
        • Main entry point for Spark functionality. A SparkContext represents the connection to a Spark cluster, and can be used to create RDDs, accumulators and broadcast variables on that cluster.
      • org.apache.spark.rdd.RDD
      • org.apache.spark.streaming.dstream.DStream
      • org.apache.spark.streaming.dstream.PairDStreamFunctions
  • Java API
    • Spark 1.6.2 Java API
    • Spark 1.2.0 Java API
  • Databricks Spark Reference Applications

• Books
  • Mastering Apache Spark
  • Faster Stateful Stream Processing in Apache Spark’s Streaming * ebruary 1, 2016)

Readings

• • Apache Storm vs. Apache Spark(2014/09)
  • Intro to Apache Spark
  • Start/Deploy Apache Spark application programmatically using spark launcher

• Performance
  • Controlling Parallelism in Spark
  • How-to: Tune Your Apache Spark Jobs (Part 1) (March 9, 2015)
  • How-to: Tune Your Apache Spark Jobs (Part 2) (March 30, 2015)

• Spark Streaming
  • Spark Streaming Example – How to Stream from Slack
  • Faster Stateful Stream Processing in Apache Spark’s Streaming (February 1, 2016)
  • How-to: Do Near-Real Time Sessionization with Spark Streaming and Apache Hadoop (November 3, 2014)

• Spark Testing
  • Spark and Spark Streaming Unit Testing (Mar 1st, 2015)
  • Spark Streaming Testing with Scala Example
  • Testing Spark: Best Practices (2014/06)

• Spark on Windows
  • Running Spark on Windows
  • winutils
    • Windows binaries for Hadoop versions
  • Getting started with Apache Spark and Scala on Windows (2 March 2016)

• misc
  • Java vs Scala vs python for big data apache spark project(2016/2/10)
  • Scala vs. Python for Apache Spark (12 Feb 2016)
    • Scala is 10 times faster than Python.
    • Scala's arcane syntax makes it difficult to master.

Integration

Kafka

• http://kafka.apache.org/
• Desc. : publish-subscribe messaging rethought as a distributed commit log.
• License : Apache License
• Written in : Scala for core, Java for clients and tools
• Sources : https://github.com/apache/kafka
• Maven artifacts : org.apache.kafka

• References
  • Official documentation
    • Kafka 0.10.0 Documentation
      • Broker Configs
    • Kafka 0.9.0 Documentation
  • API
    • Kafka 0.10.0.1 Java API
    • Kafka 0.9.0.1 Scala API

• Readings
  • Apache Kafka Tutorial
  • Embedded Kafka broker / producer / simple consumer in a single process useful for testing or for persistent queues.
  • How Can we create a topic in Kafka from the IDE using API (Jun 5 '13)
  • How to get topic list from kafka server in Java (Jul 23 '14)
  • How create Kafka ZKStringSerializer in Java?(Nov 22 '14)

Associates

CSharpClient for Kafka

• https://github.com/Microsoft/CSharpClient-for-Kafka
• Desc. : .Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes.
• Sources :

Spring for Apache Kafka

• https://projects.spring.io/spring-kafka/
• Desc. : Provides Familiar Spring Abstractions for Apache Kafka
• Sources : https://github.com/spring-projects/spring-kafka

Kafka Manager

• https://github.com/yahoo/kafka-manager
• Desc. : A tool for managing Apache Kafka.

• Readings
  • Install and Evaluation of Yahoo's Kafka Manager (3/29/2015)

Testing

ScalaTest

• http://www.scalatest.org/
• Desc. : the most flexible and most popular testing tool in the Scala ecosystem
• License :
• Maven artifacts : org.scalatest

• Readings
  • ScalaTest 2.2.6 API
    • org.scalatest.FunSuite class

Libraries

JSON

Json4s

• http://json4s.org/
• Desc. : A single AST to be used by other scala json libraries
• License :
• Sources : https://github.com/json4s/json4s
• Readings

NoSQL Client

Casbah=

• https://github.com/mongodb/casbah
• Desc. : an interface for MongoDB designed to provide more flexible access from both Java and Scala.
• License : Apache License, Version 2.0

Tools

sbt

• http://www.scala-sbt.org/
• Desc. : interactive build tool
• License :

• Readings
  • sbt 0.13 Reference Manual
  • sbt 0.13 API

Scala.js

• http://www.scala-js.org/
• Desc. : the Scala to JavaScript compiler
• License : Scala License
• Sources : https://github.com/scala-js/scala-js