nq-nqp: Lessons Learned

NQ-NQP’s Status:

  • A fairly well defined grammar for parsing nqp exists. It generates AST nodes for most of the grammar, but not all of the grammar.
    • In the future, it would be useful for AST optimizations, like constant folding
  • Code generation supports integers and subs, but it does not yet support other data types like floats, strings, bools, or anything else like regular expressions.
    • Built in basic types should eventually include:
      • Packages
      • Subs
      • Classes
      • Roles
      • Arrays
      • Hashes
      • Ints
      • Nums
      • Strings
      • Bools
  • Execution of code generated works.
    • Debugging is surprisingly easy with the gdb, but I will have to see how hard it is in the future. I know that if I start working with a JIT it becomes increasingly difficult to view whats being run on the processor.
  • A limited number of binary ops, like add and subtract are working.
  • The VM supports method calls, but the code does not currently generate code properly.

Lessons learned:

  • LLVM is neat but advanced and complicated
  • Grammars are fun, we should do more of those
    • Having a way of defining a grammar is difficult though currently, in C there is yacc/bison but that is a bit limited compared to other systems.
  • Grammars are hard, parsing is not easy
    • Tokenizers and Lexers are difficult if you want to contextually call things based off of where they should be in your grammar
  • Focusing on construction a real “Compiler” is probably a mistake. Start small, an ast that you can walk and execute each step of the way is better for learning than building a code generation system. Code generation is useful but hard, and can be deferred until later.

Things I would do different if I could start over:

  • Focus on the grammar, and making an executable AST rather than translating the AST into code.
  • If there was a VM out there I could link to for a runtime, I have not found an easy to use one. It would be nice to have because then I could of focused more on other parts of the language, but VM support is not that hard, just lots of typing.

Leave a Reply

Anti-Spam Protection by WP-SpamFree