One of my goals for the new version of Shogun was to create a language to go along with it where I used little to no reference material for the compiler. I’ve done pretty well for not having any background in creating such a compiler, if I do say so my self.
Originally, the design of Sholan (the Shogun Language compiler) made it such that imported scripts would all be compiled into a single binary; that is, the entire standard library would be included in every single program compiled by Sholan. This wasn’t exactly ideal, so I tackled the process of dynamic linking.
One of the very first issues I ran into was how to specify whether an import was an external dependency or not. The simplest way would be to simply add a keyword after the import statement which marked it as a dependency, but what about imports inside of the file you are importing? They might be separate libraries, or they might also be a part of the same import.
I solved this by adding “import modes”. There are five import modes: None, import, inherit, library, and export. When importing a new file, the file’s import mode is pushed onto a stack. When the file is not being imported anymore, then the stack is popped.
None + Import
The “None” mode and the “Import” mode, for all intents and purposes, are exactly the same. “None” simply means that the current import is the top-level file (the script originally passed to Sholan). When at one of these import modes, Sholan acts as if the code in the related file is a part of the initial file.
The “Inherit” mode is a special mode, in that it doesn’t represent anything in and of itself. It is the default mode when importing any file. When Sholan’s kernel is asked for the current import mode, it simply skips this mode and reads the mode from the next one on the stack, thereby inheriting the next import mode.
The “library” mode, as it name would suggest, is used to import a library. This tells Sholan to not compile functions inside the imported file, but instead to simply register the function symbols as usable.
The “export” mode has a special function, and is generally only to be used in build scripts (see the next section). It marks all imported functions as being exports, which is later used to setup a library for building.
Due to the architecture of the import system, building a dynamic library requires a “build script” to be written, which is a simple Sholan program that imports all library files in the “export” mode, and tells the compiler to output the correct headers for a library. This is done as follows:
#build-exports import "mylibrary.sl" export
This tells the compiler to build the export header, and then imports the library.
Linking the “wrong” way
As I said, I have no background in writing compilers, and along with that I have no knowledge of how dynamic linkage generally works. As such, I decided to tackle it in the simplest way possible. The virtual machine imports a dynamic library as normal memory, and then the program jumps to the beginning of that memory. The “export header” I talked about earlier is simply a set of operations at the beginning of each dynamic library that pushes the addresses of each function onto the stack, and then returns control to the main program. The program then stores the function addresses in memory and accesses them like it would normal variables.
The major downside to this approach is that it completely depends on the order that functions are defined inside libraries, and it also has no mechanism to prevent a library from being loaded multiple times. On the other hand, it works, and it is a good first step towards a true linkage system.
That’s all for now. Next time I might get into how the compiler works, or maybe I’ll just start some programming tutorials.