Pnuts

From Bauman National Library
This page was last modified on 8 June 2016, at 18:27.
Pnuts
Paradigm Object-oriented, Scripting
Developer Toyokazu Tomatsu (Sun Japan)
Stable release 1.2.1 / Template:Release date
Typing discipline static, dynamic, duck
Platform JVM
OS Cross-platform
License Sun Public License
Website http://java.net/projects/pnuts
Influenced by
Java

OO Programming Support in Pnuts

Over years, I was trying to understand how OO feature should be integrated into a procedural language with lexical scoping. In the recent updates of Pnuts, its class infrastructure was re-designed, based on more sophisticated internal architecture than several attempts I made in previous versions. In this article, I will explain how OO functionality has been integrated into Pnuts.

Classes in Pnuts

For example, a simple counter can be defined in Pnuts as follows.

class Counter {
  i = 0
  inc(){ i++ }}

The class is compiled into an ordinary Java class. There is no difference from Java classes once it is compiled and loaded. Attributes may or may not be typed. In the example above, the variable i represents an un-typted attribute. Getter and setter methods, which are getI() and setI() in this example, are implicitly defined for each attribute. It is assumed that class definition follows Java Beans naming convention. Getter and setter methods are called when reading (or writing) the attribute from scripts.

c = Counter()
c.i      ---> 0
c.i = 1
c.i      ---> 1

Return type and parameter types of methods are optional. If no type is specifed, Object.class is implicitly used, except when the method overrides an existing method defined in a superclass; in this case it inherits the method signature. No access control can be specified for methods and constructors. Methods and constructors are always public. The keywords this and super can be used in Pnuts as in Java. For examples, the following code illustrates how to customize a setter method.

class MyCounter extends Counter {
  setI(i){
     if (this.i != i) println("changed")
     super.setI(i)
   }
  inc(){ setI(this.i+1) }}

The following code is another example, which is a subclass of HashMap that automatically creates a Set for unmapped key.

class MyMap extends java.util.HashMap {
  get(key){
     if ((v = super.get(key)) == null){
       super.put(key, v = new java.util.HashSet())
     }
     v
  }}

Class Loading

Classes defined in Pnuts can be loaded in either of two different ways. First, script files with suffix .pnc can be automatically compiled and then loaded by PnutsClassLoader, as if it were .class files which can be loaded by system class loader. Since dependency between classes is automatically resolved by the class loader, it is less likely that linkage errors (or duplicated definition errors) occur, than the other options. Alternatively, class definition can be embedded in a script file and executed as an ordinary script file. In this case, the classes are temporarily defined in the script, that is, the scope of class name is inside the script file that contains the class definition. When script 'A' executes script 'B', those scripts uses different class loaders to load embedded classes.

Implementation

When Counter class is compiled, the compiler generates a skelton class of the specified signature. At the same time, the compiler creates a function that creates a closure. The function returns an array of functions, each one is associated with a method of the class. Attributes are mapped to local variables of the function. Getter and setter methods (and the associated functions) to access the attribute are automatically generated. For example, the following function is created for the Counter class that was previously defined.

function (this, super){
  i = 0
  [
   function inc(){ i++ },
   function getI(){ i },
   function setI(_i){ i = _i }
  ]}

Next, the class below illustrates what the skelton class looks like. It has a method and a set of fields to bind Pnuts functions. The constructor calls the function above and initializes private fields with each element of the array returned by the function. It does not contain actual code for the method body. Instead, each method just calls a Pnuts function bound to the associated private field.

public class Counter {
 static PnutsFunction function_factory;
 static Context context;
 private PnutsFunction inc_function;
 private PnutsFunction get_function;
 private PnutsFunction set_function;
 public static void attach(PnutsFunction function_factory, Context context){
     Counter.function_factory = function_factory;
     Counter.context = context;
 }
 public Counter(){
   PnutsFunction[] function_array = function_factory.call(new Object[]{this, superProxy}, context);
   this.inc_function = function_array[0];
   this.get_function = function_array[1];
   this.set_function = function_array[2];
 }
 public Object inc(){
  this.inc_function.call(new Object[]{}, context);
 }
 ...}

Since Pnuts function has lexical scoping rule, classes defined in Pnuts also have the same scoping rulle. If a class is defined in a local scope, symbols in the local scope are visible from the class definition. For example, the following function create an instance of counter, which the variable used in the inc() method comes from the parameter of the function.

function counter (i){
 new Object() {
    inc(){ i++ }
 }}

Similarly, a class attribute can be referenced from functions defined in a method.

class Counnter {
  i = 0
  inc_func(){
    {-> i++}
  }}

Lightweight Generator in Pnuts 1.1

Pnuts1.1 borrowed the idea of simple generator from Python, and is called 'lightweight generator'. It is a fundamental language features in Pnuts, along with closure and module system. The lightweight generator in Pnuts is almost as fast as function call. Some of the benefits of rewriting callback-style to generator-style are:

  • Can have control over when the callback function is called
  • Can cancel the iteration
  • Script can be simplified by replacing nested function with for-loop
  • Cleanup operation can be simplified with try-finally block in generator function.

e.g.

function safeopen(file){
    in = open(file)
    try {
        yield in
    } finally {
        close(in)
    }
 }
 for (in: safeopen(file)){
     ....   // no need to close
 }

Multiple implementations of Pnuts

Pnuts has two kinds of the language implementations in it; one is the bytecode compiler, the other is the AST interpreter. Of course, the bytecode compiler is much faster than the AST interpreter. I think that the bytecode compiler should be used in most case, but AST interpreter is sometimes needed for any of the following reasons.

  • Bytecode compiler cannot always compile scripts successfuly. For examples,
 x = 1
   x++
   x++
   ...
   (3000 times)

may fail due to the JVM limitations. Pnuts uses the AST interpreter as a fallback when the bytecode compiler throws ClassFormatError.

  • The bytecode compiler requires RuntimePermission("createClassLoader"), and RuntimePermission("getProtectionDomain"). On the other hand, the AST interpreter can run without any permissions.
  • On the AST interpreter, lexical scope can be serialized/deserialized whereas it can't on the bytecode compiler.
function counter(x){
    function inc() x++
  }
  c = counter(0)
  c()  // --> 0
  c()  // --> 1
  writeObject(c, "d:/tmp/c.ser")
  ...
  c = readObject("d:/tmp/c.ser")
  c()  // --> 2

Another benefit of having two implementations is that we have more chance of finding what is missing in the specification. For example, the semantics of variable declaration was ambiguous, more specifically, there was no description about variable declaration within conditional statements.

Control Abstraction in Pnuts

Some people say Pnuts doesn't add anything as a programming language, but that is not true. In this article, I'll show you that Generator in Pnuts can be used for control abstraction that cannot be done by other language constructs. Suppose you want to convert a ZIP file to a LHA file (or whatever archive file), without creating temporary files. In Java, you would use java.util.zip.ZipInputStream to read the ZIP entries, something like this:

 ZipInputStream zin = new ZipInputStream(in);
   ZipEntry entry;
   while ((entry = in.getNextEntry()) != null){
     ... }

Then, I would use the LHA library for Java to create LHA files. The basic usage of this API is like this:

LhaOutputStream lout = new LhaOutputStream(out);
   for (file: files){
      LhaHeader hdr = new LhaHeader(file.getName());
      lout.putNextEntry(hdr);
      lout.write(...);
      lout.closeEntry();
   }

The procedure by which an archive file is created is similar to java.util.zip.*. So, ZIP to LHA conversion can be written in Java like this:

LhaOutputStream lout = new LhaOutputStream(out);
   ZipInputStream zin = new ZipInputStream(in);
   ZipEntry entry;
   byte[] buf = new byte[512];
   while ((entry = in.getNextEntry()) != null){
      String name = entry.getName();
      long time = entry.getTime();
      LhaHeader hdr = new LhaHeader(name);
      hdr.setLastModified(new Date(time));
      lout.putNextEntry(hdr);
      while ((nread = zin.read(buf)) != -1){
         lout.write(buf, 0, nread);
      }
      louut.closeEntry();
   }
   zin.close();
   lout.close();

In Pnuts, I would use readZipEntries() and writeLhaEntries() to convert a ZIP file to a LHA file. use("lha")

   function zip2lha(infile, outfile){
     function entries(zfile){
      for (e:readZipEntries(zfile)){
        yield {"header"=>{"path"=>e.entry.name, "lastModified"=>date(e.entry.time)},
               "input"=>e.input}
      }
     }
     writeLhaEntries(outfile, entries(infile)) }

So what's the difference between Java code above and the Pnuts script? The most important difference is that the control to call getNextEntry() and the control to call putNextEntry() are clearly separated and the both are provided as reusable functions; i.e. readZipEntries() and writeLhaEntries().

Invokedynamic

As an implementor of Pnuts, I have to write about JSR292, a.k.a invokedynamic. My conclusion is that invokedynamic definitely won't help. If I were a member of EC, I'd vote NO, because it is not worth adding a new mnemonic to the VM spec. Even if the capability is provided as an API, it will rarely be used by JVM languages. (I'm not talking about hotswapping here.) Pnuts is one of the fastest scripting language on JVM. One secret of the efficient implementation is sophisticated method caching. Pnuts usually does not use Reflection API to call methods, even when accessing JavaBeans properties. Pnuts generates method proxies on-the-fly and reuse them. Reflection API is used only when method proxy can't be used because of classloader mismatch. It is true that searching appropriate methods from actual arguments is expensive, but the impact is almost completely invisible, thanks to the method cache. In fact, method call in Pnuts is even faster than that of Ruby. Is it still worth adding a new mnemonic? Or the mnemonic is for other language implementors? In Pnuts, it is possible to give a hint to select a particular method, assuming that the method is overloaded and two or more methods match the actual arguments. When some of the actual arguments have "cast" to some type, the information is used to choose an appropriate method. e.g. String.valueOf((char[])array). How can it be achieved using invokedynamic? Also, Pnuts searches methods through JavaBeans method descriptors, but not Class.getMethods(..). And the way of searching methods/constructors can be customized. Under some configuration, private methods can be selected. This kind of flexibility can be achieved only at application-level. The cause of slow down is not just invoking methods. Some other JVM languages don't generate bytecode and interpret the code in their implementation. They can't be faster unless a bytecode compiler is implemented. Even JVM languages with bytecode compiler, the generated code and the runtime library should be efficient. If the generated code and the runtime library is not efficient, invokedynamic would have no effect.

References