Scala random notes, mostly on back-translating to Java
March 2017
last update:
My need for Scala understanding is hardly out of curiosity. The more I look at
it, the less I am thrilled to get on any bandwagon. This also characterizes my
attitude generally about functional programming. It's popular to rave about it;
I see it as more difficult to grok when you inherit someone's code and generally
pointless as I don't see it living up to its claimed goals of immutability and
readability. The complained-of Java boiler plate or verbosity is somewhat resolved
by better modularity, methods, superclassing, etc. However, just as heavily
@ -annotated frameworks annoy me because I can't tell what's going on,
I'd rather crawl through a pile of Java code that's an honest portrayal of
reality than guess at someone else's mind.
I once was called upon to back-translate some Scala code to Java, which I did
willingly since what that code accomplished had high value to the project I was
on. These are my notes on back-translating and on Scala topics that I needed
to explore in order to understand what I was doing. These notes aren't exhaustive
at all since I gained more of a feeling for how to translate than clear understanding
from which I could scratch out sure methodologies. Sorry.
A growing worry I think I have confirmed is that Scala can create inter-class
relationships that cannot be easily back-translated directly into Java. The
code must be rewritten a different way. I don't see this as superiority in Scala,
just as a salient difference. It's like this when translating between any two
programming (or even natural, spoken) languages.
General notes
Translating Scala code to Java is a little like shopping for furniture from
a store to decorate an empty living room based on what you're looking at in
a Picasso painting.
There are no Scala-to-Java translators. Indeed, it may be impossible ultimately
to write such a thing. There are some Java-to-Scala translation helps. So...
what? UML?
I looked into a UML rendering of Scala code. Sadly, the first (and last, so far)
time I used UML in IntelliJ, I did not take notes on how to do it. It appears,
however, that there is no plug-in for Scala UML for IntelliJ (while there is for
Java). There is
a project of Scala code
that generates DOT at the command line, but you have to use
Graphviz , a bit of open-source software
that would take quite a bit of effort to figure out how to use. I didn't have
time to screw around with all of that. There is an Eclipse plug-in, but it's
for generating Scala stubs based on a given model and not for generating UML
diagrams from existing Scala code.
So, it appears that there is no help for rendering Scala code in any
representational format.
It's generally good to take time out to catch back up on Java 8 constructs
because they're (sort of) close to the Scala ones. This is my advice.
Sloppy Scala programming
I had opportunity to engage in a cordial conversation with some Scala guys at
the UJUG. I asked them about some troubles I was having and described classes in
which code statements were interwoven with method definitions. In Java, of course,
that's syntactically impossible. They confirmed that such statements would be
part of the initialization of the class in particular. Their opinion was also
that this was not good Scala (to toss them around instead of grouping them all
near the top of the class.
Scala traits
These are similar to Java's interface . They share interfaces and fields
between classes. Classes and objects can extend traits, but like interface s
and abstract class es, traits cannot be instantiated (nor have parameters).
Scala's case class
This is for modeling classes containing immutable data. They are used
especially for pattern matching. They have an implicit apply() method
such that you construct new additions to them as if constructing an instance of
the class:
case class Book( isbn: String )
val frankenstein = Book( "978-0486282114" )
The -> operator
The -> (or goes-to) operation is shorthand for associating a key
(left-hand side of the operator) with a value (right-hand side). This is instead
of using the constructor or the put() method of the map class.
=> is syntactic sugar for creating instances of functions. In Scala,
every function is an instance of a class. For example,
Int => String
...is equivalent to
Function1[ Int, String ]
So, this is a function that takes and argument of type Int and returns a
String .
One stackoverflow responder made a suggestion that (and just how accurate it is
I don't know, but I sort of see his point) a simplified way of thinking about
this "operator" is to think of what you're looking at as:
LEFT => RIGHT
...means, take (what's on the) LEFT and instead do (what's on the)
RIGHT ." Or, "given LEFT , do RIGHT ."
Scala's _ (underscore) operator
When I'm finished looking through these, I'm just as confused if not more. One
read that's sometimes useful is
Scala _ [underscore] magic .
import xyz._
Wild card—all of xyz is imported
import xyz.{ Predef => _ , _ }
Exception, everything except Predef
def f[ M[ _ ] ]
Higher kinded type parameter
def f( m: M[ _ ] )
Existential type
_ + _
Anonymous function placeholder parameter
m _
Eta expansion of method into method value
m( _ )
Partial function application
_ => 5
Discarded parameter
case _ =>
Wild card pattern—matches anything
val ( a, _ ) = ( 1, 2 )
ibid
for ( _ <- 1 to 10 )
ibid
f( xs: _ * )
Sequence xs is passed as multiple parameters to f( ys: T* )
case Seq( xs @ _ * )
Identifier xs is bound to the whole matched sequence
var i: Int = _
Initialization to the default value
def abc_ <>!
An underscore must separate alphanumerics from symbols on identifiers
t._ 2
Part of a method name, such as tuple getters
Scala's ternary operator
Scala doesn't implement the C/Java ternary operation. However, it does offer:
if( condition ) true-result else false-result
Scala icons in IntelliJ IDEA
"Companion" objects
...may be found at the bottom (?) of Scala classes and are "analogous to Java
having a class with static methods." At the bottom of Stratio's Lucene plug-in
code for Cassandra, for instance, class IndexQueryHandler .
The highlighted lines and following constitute this companion object:
/*
* Copyright (C) 2014 Stratio (http://stratio.com)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.stratio.cassandra.lucene
.
.
.
class IndexQueryHandler extends QueryHandler with Logging
{
.
.
.
}
/** Companion object for [[IndexQueryHandler]]. */
object IndexQueryHandler
{
val processResults: Method = classOf[SelectStatement].getDeclaredMethod( "processResults",
classOf[PartitionIterator],
classOf[QueryOptions],
classOf[Int],
classOf[Int] )
processResults.setAccessible( true )
/** Sets this query handler as the Cassandra CQL query handler, replacing the previous one. */
def activate(): Unit =
{
this.synchronized
{
if( !ClientState.getCQLQueryHandler.isInstanceOf[IndexQueryHandler] )
{
try
{
val field = classOf[ClientState].getDeclaredField( "cqlQueryHandler" )
field.setAccessible( true )
val modifiersField = classOf[Field].getDeclaredField( "modifiers" )
modifiersField.setAccessible( true )
modifiersField.setInt( field, field.getModifiers & ~Modifier.FINAL )
field.set( null, new IndexQueryHandler )
}
catch
{
case e:
Exception => throw new IndexException( "Unable to set Lucene CQL query handler", e )
}
}
}
}
}
A good way to translate that companion object into Java is to make (in this case,
the two methods) simply static:
.
.
.
public class OurQueryHandler implements Logging, QueryHandler
{
.
.
.
public static Method processResults()
{
try
{
Class< ? > clazz = SelectStatement.class;
Method method = clazz.getDeclaredMethod( "processResults", PartitionIterator.class,
QueryOptions.class,
int.class, // (strange, but true!)
int.class );
method.setAccessible( true );
return method;
}
catch( NoSuchMethodException e )
{
throw new IndexException( "Unable to return reflection of method SelectStatement.processResults()", e );
}
}
public static void activate()
{
synchronized( OurQueryHandler.class )
{
if( !( ClientState.getCQLQueryHandler() instanceof OurQueryHandler ) )
{
try
{
Class< ? > clazz = ClientState.class;
Field methodField = clazz.getDeclaredField( "cqlQueryHandler" );
Field modifiersField = Field.class.getDeclaredField( "modifiers" );
methodField.setAccessible( true );
modifiersField.setAccessible( true );
modifiersField.setInt( methodField, methodField.getModifiers() & ~Modifier.FINAL );
methodField.set( null, new OurQueryHandler() );
}
catch( Exception e )
{
throw new IndexException( "Unable to set our CQL handler", e );
}
}
}
}
}
Unit in Scala
Unit is analogous to void.
asScala
This allows you to take
List< String > javaStringList = Array.fromList( "this", "that", "and", "the", "other", "thing" );
...and turn this list into a Scala construct that will function using map()
and other functionality:
val scalaStringList: List[String] = javaStringList.asScala
This permits, from this point on, the use of Scala constructs like:
scalaStringList.forEach{ s => ... }
Seq and List in Scala
The following may be a bit confusing...
In Java terms, Scala's Seq would be Java's List , and Scala's
List would be Java's LinkedList . However, I have often rendered
Scala's List as merely Java's List .
Note that Seq is a trait , which is equivalent to Java's
interface , but with the equivalent of up-and-coming defender methods.
Scala's List is an abstract class that is extended by Nil and
:: , which are the concrete implementations of List .
So, where Java's List is an interface , Scala's List is
an implementation.
Multiple .map() calls in initialization
Here's a construct by way of example. There's an instance variable, pools ,
in a class named TaskQueue . Once initialized, this variable points at
a list of task-queue pools. This class is constructed with the number of threads
to support. The interspersed comments below recount what's going on.
private val pools =
// Start with a range from 1 to numThreads, which you can essentially
// consider a Seq(1, 2, 3, ..., numThreads).
( 1 to numThreads )
// Now, for each value in that range, pass it to this function and store
// the results in another sequence. The function ignores the input value
// (which is what the _ means before the arrow) and returns a
// new ArrayBlockingQueue[Runnable]. After this step, we will have
// a Seq[ArrayBlockingQueue[Runnable]], with a number of elements
// equal to numThreads.
.map( _ => new ArrayBlockingQueue[Runnable]( queuesSize, true ) )
// Again, for each of these ArrayBlockingQueues, pass it to this
// new function (captured in the q argument) and create a new Seq
// of the results. In this case, it is a Seq[ThreadPoolExecutor]
// where each of the ThreadPoolExecutor objects has been through the
// initialization code below
.map( q => new ThreadPoolExecutor( 1, 1, 1, DAYS, q,
new BasicThreadFactory.Builder()
.namingPattern( "indexer-%d" )
.build(),
( task, executor ) =>
if( !executor.isShutdown )
executor.getQueue.put( task ) ) )
// The final resulting type of pools should be Seq[ThreadPoolExecutor],
// and it will have a number of element equal to numThreads.
In Java, this might be something like this:
List< ThreadPoolExecutor > pools = new ArrayList<>();
BasicThreadFactory threadFactoryBuilder = new BasicThreadFactory.Builder()
.namingPattern( "indexer-%d" )
.build();
// create a list...
List< Queue< Runnable > > queues = new ArrayList<>( numThreads );
// ...and fill it with new queues of queueSize and our default access policy...
while( numThreads-- > 0 )
queues.add( new ArrayBlockingQueue<>( queueSize, DEFAULT_ACCESS_POLICY ) );
// fill the list of executors with initialized executors that use, each, a queue from above...
for( Queue< Runnable > queue : queues )
pools.add( new ThreadPoolExecutor( CORE_POOL_SIZE, MAX_POOL_SIZE, KEEP_ALIVE_TIME, KEEP_ALIVE_FOR_DAY,
( BlockingQueue< Runnable > ) queue, threadFactoryBuilder ) );
Second example
The objective is to build a list of FsIndex .
private[this] val indexes: List[FsIndex] = partitions match
{
case 1 =>
List( new FsIndex(name, path, analyzer, refreshSeconds, ramBufferMB, maxMergeMB, maxCachedMB ) )
case n if n > 1 =>
val root = path.toFile.getAbsolutePath + File.separator
( 0 until n )
.map(root + File.separator + _ )
.map(Paths.get( _ ) )
.map(new FsIndex( name, _, analyzer, refreshSeconds, ramBufferMB, maxMergeMB, maxCachedMB ) )
.toList
case _ =>
throw new IndexException( s"The number of partitions should be strictly positive but found $partitions" )
}
if( numPartitions < 1 )
throw new IndexException( "The number of partitions should be greater than 0" );
List< FsIndex > indices = new ArrayList<>();
final String ROOT = path.toFile().getAbsolutePath();
switch( numPartitions )
{
case 1 :
indices.add( new FsIndex( name, path, analyzer, refreshSeconds, ramBufferMb, maxMergeMb, maxCachedMb ) );
break;
default :
// build a new index for each set of data we can muster here...
for( int partition = 0; partition < numPartitions; partition++ )
{
indices.add( new FsIndex( name,
Paths.get( ROOT + File.separator + partition ),
analyzer, refreshSeconds, ramBufferMb, maxMergeMb, maxCachedMb ) );
}
break;
}
Operators /: (foldLeft() ) and :\ (foldRight() )
This is pretty crazy stuff. I read one post in which the guy tried very hard to
elucidate this in this way:
Think of /: as a picture of a domino falling right into the first wagon
of a train of other dominoes—the way many set them up to see them knock
each other over. Every time the calculation-so-far makes contact with a new
element in the train, the function is applied.
val nums = List( 4, 2, 9, 3, 1 )
val sum = ( 0 /: nums ) ( _+_ )
_ + _ _ _ _ _
/ / | | | | | | | | | |
/ / | | | | | | | | | |
/0/ |4| |2| |9| |3| |1|
/ / | | | | | | | | | |
/ / | | | | | | | | | |
\____________nums___________/
_ + _ _ _ _
/ / | | | | | | | |
/ / | | | | | | | |
0 + 4 = /4/ |2| |9| |3| |1|
/ / | | | | | | | |
/ / | | | | | | | |
_ + _ _ _
/ / | | | | | |
/ / | | | | | |
(0 + 4) + 2 = /6/ |9| |3| |1|
/ / | | | | | |
/ / | | | | | |
_ + _ _
/ / | | | |
/ / | | | |
(0 + 4 + 2) + 9 = 15/ |3| |1|
/ / | | | |
/ / | | | |
_ + _
/ / | |
/ / | |
(0 + 4 + 2 + 9) + 3 = 18/ |1|
/ / | |
/ / | |
_
/ /
/ /
(0 + 4 + 2 + 9 + 3) + 1 = 19/
/ /
/ /
Another example
( 0L /: indices ) ( _ + _.getNumDocs )
In this example, what's done is the values of getNumDocs() as called
on the various indices of a list, with this effect:
List< FsIndex > indices = [ ... ];
long count = 0;
for( FsIndex index : indices )
count += index.getNumDocs();