HFOSS: What Is Open Source and How Does It Work?

For HFOSS, I had to read a chapter (3) of Steve Weber’s The Success of Open Source. It gives a nice shortened history of the creation of Linux as an anecdote for how open source software projects grow. The chapter talks about how OSS projects are structured, the upsides and downsides of letting a community govern a large project, and who participates in the projects (hint: it’s not just the developers, you also have everyone who uses the software). Some studies are shown that try to dissect what kind of people contributed to the Linux project (by country, by what email they use, etc). The chapter goes on to explain the goals of OSS and how the projects are run: Don’t reinvent the wheel (share what you made so others can benefit), create opportunities for developers to participate in projects, and to develop solutions to problems by distributing the work.

I’d say that this chapter gives a good overview of how OSS is created and why it exists, though I would say that the chapter is a bit outdated (it mentions SourceForge as a collaboration site that is commonly used, but nowadays SF is the last place you would want to host a project). I’d recommend reading over the chapter if you don’t have much knowledge about open source software, but at the same time if you already actively participate in OSS communities there is probably not much within the chapter that you don’t already know.

Some thoughts about not-really-open-source engines

Today I heard about Amazon releasing a game engine known as Lumberyard, which is really just an offshoot of Cryengine. The license is interesting – it gives full access to the source code for free. Unreal Engine 4 has a similar license, which allows access to the full UE4 source code.

Not OSS

Thing is, despite both engines allowing full access to source code, they aren’t really open source software. Unreal has a weird bit of its license that says you can’t use any source code from the editor components in the game itself. This seems to be to prevent someone from creating a standalone editor for users without needing them to accept Epic’s licensing terms. Both engines restrict who you can redistribute source code to (only others who have also accepted the license).

The bigger tradeoff between their licenses, however, is how each engine makes money. Lumberyard has zero royalties, but if you want to use a cloud service to host game servers you are required to use Amazon AWS (though they have an exception if you want to use your own hardware). Unreal has no such stipulations, but Epic does require a 5% royalty on gross profit (removing that royalty requires a special and very expensive license from Epic Games).

Community Participation

The biggest problem I see with Lumberyard versus Unreal Engine is that Lumberyard doesn’t seem to have a way for the community to participate in its development. That’s a huge part of OSS, and while Unreal isn’t OSS it does get the closest I’ve seen for a piece of commercial software: Once you have accepted Epic’s license terms, you can get access to the Unreal Engine github project where the community can and does submit patches to the engine itself all the time.

Conclusion

Despite the offhand use of “open source software” as a label for UE4 and Lumberyard on forums (see: reddit), they are both definitely not in the spirit of open source, though that certainly won’t prevent me from using them. If anything, what’s going to prevent me from using Lumberyard is the fact that right now it’s pretty much just Cryengine under a different license, and usability of Cryengine leaves something to be desired. I’ll probably be sticking with Unreal for the time being.

HFOSS: First Flight

This semester, I am enrolled in a class that seems strangely situated at RIT: Humanitarian Free and Open Source Software which is set to be an introduction to the world of HFOSS. Now, I’m not unfamiliar with said world (I administer numerous linux systems, open source much of my work, and use quite a bit of OSS myself) but I figured it might be interesting to take the course. I say the course is strangely situated, as the class is offered through the department of Interactive Games and Media, rather than the CS or SE department like I’d expect.

This week’s work was short & sweet, as it is only the first week of classes. The assignment was to add a yaml file describing who I am and where my github/bitbucket/etc profiles are. The file was submitted via a github pull-request (I admit I was a bit lazy here; I didn’t clone the repository, I just used github’s built in editor). Only problem was the only example file to look at actually had the “forges” section (which lists profiles for sites like github) setup wrong, so I ended up having to go back and redo it. I actually haven’t used github’s pull request system too much myself (though I am fairly familiar with it) if only just because most of the projects I work on are personal projects and thus never make use of the system even if they exist on github.

That’s all for now!

Open Sourcing PlayPen

The Chunk, where I am sadly leaving, is currently in the process of being merged with CubeCraft. As part of the merger, we are getting to do whatever we want with tech that isn’t going to be used. PlayPen, my server management framework that I originally wrote for managing the large number of minecraft servers we have, is some of that tech. As such, it is being open sourced here.

PlayPen is a generic server management framework, made for managing ephemeral services (ones that do not need to store data across restarts and can be stopped/started at will) across large numbers of servers. While we used it to provision and manage minecraft servers, it doesn’t assume anything about what you are trying to run and is especially useful for running practically any kind of game server.

Just a Framework

PlayPen can’t do much by itself, and only provides a framework for managing servers. It can provision/deprovision services but needs something to tell it what to do and when to do it. To do so, you either need an external program that can talk with playpen (such as the command line client or PVI). For that purpose, we developed MCBalancer for minecraft servers, which automatically creates and destroys minecraft instances depending on how many players are online. It will be pushed to our repositories soon (needs some cleaning up).

Reliability

There are quite a few issues to be fixed, mostly for the sake of cleaning up the codebase (and network protocol – that’s a bit of a mess), but PlayPen is stable enough that we’ve had local coordinators (the parts of playpen that actually run services) run for months on end without restarts. The only times we needed to restart them was to add new features that we needed for newer services.

The Future

I have a lot of plans for playpen and I plan to make it much easier to run services that don’t fit the ephemeral model as well. Other ideas include the ability to control docker containers in addition to normal playpen services. Further plugin support (like allowing them on local coordinators) and some new official plugins (REST API plugin so you don’t have to work with the network protocol directly? Yes please!) should help out a lot. Maybe even a more generic load balancing system that replaces MCBalancer and allows more than just minecraft servers to work.

Making image transformations fun

A month or two ago, I noticed a post on Hacker News (or was it reddit?) about pixQL, which provides an sql-like syntax to query and create images. I was intrigued, and thought it was a really cool idea. A simple scripting language that made it easy to do programmatic transformations on images.

I decided I wanted to create my own take on this idea. It wasn’t that I thought I could do better than pixQL, or that I thought anything was missing from pixQL. I just really like designing languages and then implementing them from scratch. I picked C# to implement my new language, with plans to move it to C++ after getting the idea fleshed out. I named the project ImageQuery.

The initial version of ImageQuery is written in C# and made up of two main components: The ImageQuery library, which provides an API to access the language, and IMQ, which acts as a command-line frontend.

Image Query Language

I named the language that ImageQuery uses IQL, or the Image Query Language. Here’s a quick sample of what it looks like:

input in
output out[in.width, in.height]

param num spacing = 5
param num size = 3
param color gridColor = {0}

out: select color from in where x % spacing < size and y % spacing < size else gridColor

Those 8 lines of code (1 if you ignore the boilerplate input/output definitions and don’t allow specifying parameters) create a grid pattern on top of the input image. For example, here we have an input:

infinity

 

Now let’s add a blue grid:

IMQ.exe -i in infinity.jpg -o out out.png -d gridColor "{0,0,1}" grid.iql

The output:

grid output sample

 

The command is fairly simple: It defines the input and output, some parameters, and the IQL file to run. IMQ also has the option to run as an interactive shell, which can be turned on with the -c or -interactive options.

Fun Stuff

I’m going to end here, but not before dropping a quick IQL script for you to try. See if you can guess what it does before running it!

Source Code (Github)

input in
output out[in.height,in.width]

out[y,x]: select {(r + g + b)/3} from in

xtlbot: twitch moderation in Lua

A few weeks ago I promised a writeup of  xtlbot, which is a twitch moderation bot written in Lua. While it might not be as feature packed as NightBot or MooBot (for example, it has no web interface), it is open source. Even without modifying the core sourcecode, you can easily add new functionality through the use of plugins.

Features

  • Easy to use plugin api
  • Permission system
  • Easy to customize
  • Open source (MIT)

Setup

Setting up xtlbot is fairly easy. You first need to make sure you have luarocks + lua 5.1 installed. You can then run these commands to install all the dependencies:

luarocks install irc-engine
luarocks install lsqlite3
luarocks install luasocket
luarocks install luafilesystem

Next, you can run the configuration script:

lua configuration.lua

You might have to use lua5.1 in place of the lua command depending on your installation.

The configuration script will ask you for details about the user xtlbot should run as along with the name of the channel xtlbot should participate in. It will then ask for the name of the twitch user it should set as the first administrator. Finally, it will ask you which plugins it should enable initially. You’re ready to start xtlbot! Run this command:

lua xtlbot.lua

The bot will start listening to your channel. You can test it by running !ping to see if you get a response.

Creating Plugins

xtlbot’s core can’t do much by itself. The majority of functionality is provided through the use of plugins. Plugins that come with xtlbot include polls, raffles, a chat filter, timed announcements, custom commands, and a basic points system.

xtlbot’s plugin api is incredibly simple, making it really easy to write new plugins. For example, this is all it takes to send a message every 30 seconds:

local core = require("src.core")
local lang = require("src.lang")
local socket = require("socket")

local plugin = {}

local last_message_time = 0

local function message_loop()
    local currentTime = socket.gettime()
    if currentTime > last_message_time + 30 then
        core.send(lang.myplugin.message)
        last_message_time = currentTime
    end
end

function plugin.init()
    core.hook_loop(message_loop)
end

return plugin
local lang = {
  message = "Hello twitch!"
}

return lang

This also demonstrates the usage of the message system (lang) to make it easy for users of xtlbot to customize messages in the config files.

More complex plugins are also possible. xtlbot contains a few functions for plugins to hook into (for example, the hook_loop used above) and an sqlite3 database for storing data.

Adding a Command

Here’s a quick example of how to create a command:

local core = require("src.core")
local lang = require("src.lang")
local commands = require("src.commands")

local plugin = {}

local function cmd_hello(user, args)
    core.send_to_user(user.name, lang.mycommand.hello)
end

function plugin.init()
    commands.register("hello", "say hi", cmd_hello, nil) -- we use nil for the last argument to say everyone can use this command.
                                                         -- if you want to restrict the command, then put a string containing the
                                                         -- name of the permission in place of nil.
end

return plugin
local lang = {
  hello = "Hi there!"
}

return lang

Enabling Plugins

Something I neglected to mention was how to actually enable plugins. Other than just placing the plugin files in their corresponding directories, you need to also add the name of the plugin to the table in config/plugins.lua. Once you’ve done that, you can run xtlbot and it will have all the plugins you specified!

Are we there yet?

xtlbot works, and is pretty hard to crash or break (all commands and hooks throwing errors won’t crash the bot), but it hasn’t been tested on a large channel yet. Additionally, a single installation of xtlbot can only be run on a single channel at a time (running two instances from the same installation will cause issues with sqlite3).