Language
- The Rise and Fall of Scala (Oct. 06, 16)
Grammar
- Scala _ (underscore) magic (29 Mar 2010)
- Quick Explanation of Scala’s (_+_) Syntax (9 Jan 2008)
Collection
- Scala’s Collections Library
- Arrays
- Convert from
scala.collection.Seq<String>
tojava.util.List<String>
in Java code (Jul 19 '13) - Five ways to create a Scala List (October 27 2016)
- Lisp style(
1 :: 2 :: 3 :: Nil
),List(1,2,3)
,List.range(1, 3)
, ...
- Lisp style(
Concurrency
Logging
- error with slf4j: ambiguous reference to overloaded definition (21 Dec 2012)
logger.debug("hello {} / {} ", "Hello", "World":Any)
External Process
scala.sys.process
packagescala.sys.process.ProcessBuilder
trait- How to download URL contents to a String or file (June 3 2016)
new URL("http://google.com") #> new File("Output.html") !!
- How to execute (exec) external system commands in Scala (June 3 2016)
Functional Programming
- A Beginner-Friendly Tour Through Functional Programming in Scala (Dec. 13, 16)
- Higher Order Functions
misc
- Java + Scala
- Start/Deploy Apache Spark application programmatically using spark launcher (June 26, 2015)
Null
,null
,Nil
,Nothing
,None
, andUnit
in Scala (July 12, 2009)
Frameworks
IoC
Spring Scala
- http://hub.darcs.net/psnively/spring-scala
- Desc. : make it easier to use the Spring framework in Scala.
- License :
- Sources
MVC
Play
- https://www.playframework.com/
- Desc. : The High Velocity Web Framework For Java and Scala
- License
Persistency
MyBatis Scala
- http://www.mybatis.org/scala/
- Desc. :
- License
Networking
Finagle
- http://twitter.github.io/finagle/
- Desc. : an extensible RPC system for the JVM, used to construct high-concurrency servers.
- License :
- Written in : Scala
- Sources : https://github.com/twitter/finagle
- Readings
Distributed
- Akka vs Storm (25 June, 2013)
Akka
- http://akka.io/
- Desc. : a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM.
- License : Apache 2 License
- Written in :
- Sources : https://github.com/akka/akka
- References
- Akka 2.4.4
- Akka 2.3.15
- Typesafe Config Library API
- static
com.typesafe.config.ConfigFactory.load()
method - static
com.typesafe.config.ConfigFactory.defaultApplication
methodapplication.conf
,application.json
,application.properties
,config.file
,config.resource
, andconfig.url
- static
- Readings
- Actor model (Wikipedia)
- Understanding actor concurrency (Feb 24, 2009)
- key principles of actor model
- No shared state
- Lightweight processes
- Asynchronous message-passing
- Mailboxes to buffer incoming messages
- Mailbox processing with pattern matching
- key principles of actor model
- Actor Path
- An actor path consists of an anchor, which identifies the actor system, followed by the concatenation of the path elements, from root guardian to the designated actor; the path elements are the names of the traversed actors and are separated by slashes.
- Getting Started Tutorial (Java): First Chapter (for Akka 2.0.2)
- Building a scalable and highly available reactive applications with Akka! Load balancing revisited(July 15, 2014)
- Why We Picked AKKA Cluster as our Microservice Framework
- Discovering message flows in actor systems with the Spider Pattern
- Blocking Needs Careful Management
- The non-exhaustive list of adequate solutions to the “blocking problem” includes the following suggestions:
- Do the blocking call within an actor (or a set of actors managed by a router [Java, Scala]), making sure to configure a thread pool which is either dedicated for this purpose or sufficiently sized.
- Do the blocking call within a Future, ensuring an upper bound on the number of such calls at any point in time (submitting an unbounded number of tasks of this nature will exhaust your memory or thread limits).
- Do the blocking call within a Future, providing a thread pool with an upper limit on the number of threads which is appropriate for the hardware on which the application runs.
- Dedicate a single thread to manage a set of blocking resources (e.g. a NIO selector driving multiple channels) and dispatch events as they occur as actor messages.
- The non-exhaustive list of adequate solutions to the “blocking problem” includes the following suggestions:
- Akka Cluster Samples with Java
- Akka Camel Samples with Java (April 14, 2016)
- Akka Java Spring
- Akka cluster: how to disable ClusterHeartbeat log (Apr 28 '15)
- Akka Utils: get a single ActorRef for a path
- Routing
- AkkaRouterExample
- Core concepts for routing
akka.routing.Router
: default and only router implementation, not inheritable and immutableakka.routing.RouterConfig
: router factoryakka.routing.RoutingLogic
: routing logic, should be thread-safeakka.routing.Routee
: destination for messages routed via a router
- built-in Akka routers and alias
akka.actor.router.type-mapping {
from-code = "akka.routing.NoRouter"
round-robin-pool = "akka.routing.RoundRobinPool"
round-robin-group = "akka.routing.RoundRobinGroup"
random-pool = "akka.routing.RandomPool"
random-group = "akka.routing.RandomGroup"
balancing-pool = "akka.routing.BalancingPool"
smallest-mailbox-pool = "akka.routing.SmallestMailboxPool"
broadcast-pool = "akka.routing.BroadcastPool"
broadcast-group = "akka.routing.BroadcastGroup"
scatter-gather-pool = "akka.routing.ScatterGatherFirstCompletedPool"
scatter-gather-group = "akka.routing.ScatterGatherFirstCompletedGroup"
tail-chopping-pool = "akka.routing.TailChoppingPool"
tail-chopping-group = "akka.routing.TailChoppingGroup"
consistent-hashing-pool = "akka.routing.ConsistentHashingPool"
consistent-hashing-group = "akka.routing.ConsistentHashingGroup"
}
Spark
- https://spark.apache.org/
- Desc. : a fast and general engine for large-scale data processing.
- License :
- Written in : Scala
- Maven artifacts : org.apache » spark
- References
- Official documentation
- Scala API
- Spark 2.0.0 Scala API
- Spark 1.6.2 Scala API
org.apache.spark.SparkConf
- Configuration for a Spark application. Used to set various Spark parameters as key-value pairs.
org.apache.spark.SparkContext
- Main entry point for Spark functionality. A SparkContext represents the connection to a Spark cluster, and can be used to create RDDs, accumulators and broadcast variables on that cluster.
org.apache.spark.rdd.RDD
org.apache.spark.streaming.dstream.DStream
org.apache.spark.streaming.dstream.PairDStreamFunctions
- Java API
- Databricks Spark Reference Applications
- Books
Readings
- Performance
- Controlling Parallelism in Spark
- How-to: Tune Your Apache Spark Jobs (Part 1) (March 9, 2015)
- How-to: Tune Your Apache Spark Jobs (Part 2) (March 30, 2015)
- Spark Streaming
- Spark Testing
- Spark and Spark Streaming Unit Testing (Mar 1st, 2015)
- Spark Streaming Testing with Scala Example
- Testing Spark: Best Practices (2014/06)
- Spark on Windows
- Running Spark on Windows
- winutils
- Windows binaries for Hadoop versions
- Getting started with Apache Spark and Scala on Windows (2 March 2016)
- misc
- Java vs Scala vs python for big data apache spark project(2016/2/10)
- Scala vs. Python for Apache Spark (12 Feb 2016)
- Scala is 10 times faster than Python.
- Scala's arcane syntax makes it difficult to master.
Integration
Kafka
- http://kafka.apache.org/
- Desc. : publish-subscribe messaging rethought as a distributed commit log.
- License : Apache License
- Written in : Scala for core, Java for clients and tools
- Sources : https://github.com/apache/kafka
- Maven artifacts :
org.apache.kafka
- References
- Official documentation
- API
- Readings
- Apache Kafka Tutorial
- Embedded Kafka broker / producer / simple consumer in a single process useful for testing or for persistent queues.
- How Can we create a topic in Kafka from the IDE using API (Jun 5 '13)
- How to get topic list from kafka server in Java (Jul 23 '14)
- How create Kafka ZKStringSerializer in Java?(Nov 22 '14)
Associates
CSharpClient for Kafka
- https://github.com/Microsoft/CSharpClient-for-Kafka
- Desc. : .Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes.
- Sources :
Spring for Apache Kafka
- https://projects.spring.io/spring-kafka/
- Desc. : Provides Familiar Spring Abstractions for Apache Kafka
- Sources : https://github.com/spring-projects/spring-kafka
Kafka Manager
- https://github.com/yahoo/kafka-manager
- Desc. : A tool for managing Apache Kafka.
- Readings
- Install and Evaluation of Yahoo's Kafka Manager (3/29/2015)
Testing
ScalaTest
- http://www.scalatest.org/
- Desc. : the most flexible and most popular testing tool in the Scala ecosystem
- License :
- Maven artifacts :
org.scalatest
Libraries
JSON
Json4s
- http://json4s.org/
- Desc. : A single AST to be used by other scala json libraries
- License :
- Sources : https://github.com/json4s/json4s
- Readings
NoSQL Client
Casbah=
- https://github.com/mongodb/casbah
- Desc. : an interface for MongoDB designed to provide more flexible access from both Java and Scala.
- License : Apache License, Version 2.0
Tools
sbt
- http://www.scala-sbt.org/
- Desc. : interactive build tool
- License :
- Readings
Scala.js
- http://www.scala-js.org/
- Desc. : the Scala to JavaScript compiler
- License : Scala License
- Sources : https://github.com/scala-js/scala-js
Community content is available under CC-BY-SA unless otherwise noted.