Test My Code: An Automatic Assessment Service for the Extreme Apprenticeship Method - 2nd International Workshop on Evidence-Based Technology Enhanced Learning - page 109

the existing systems did fulfill some of our requirements, perhaps the closest match
being Marmoset [15]. Despite testing several candidates, we did not find a system
that would suit our specific purposes. Having just an assessment service would waste
tons of experience that we have gathered while scaffolding students: surely some of
the experience could be built into a system.
Work on Test My Code (TMC) started in 2011 as two bachelor-level capstone-
projects, where one team was assigned to work on an assessment server, and one
team on an IDE plugin. During the projects, the teams built a working prototype,
which has since been further developed into a flexible assessment service that inte-
grates seamlessly into the programmers' workflow. TMC can be deployed to cloud
environments, and it supports the creation of exercises with built-in scaffolding.
TMC is based on client-server architecture, see Figure 1. On the client side the
user has a NetBeans plugin that: (1) retrieves and updates the exercises from a server,
(2) provides scaffolding and runs tests locally, (3) submits exercises to the server,
and (4) gathers snapshots from the students' working process. Snapshot gathering
can be enabled or disabled by the student. In addition to the plugin, TMC has a
web interface that students use for creating and administering their user account as
well as viewing suggested solutions. Course instructors use the web interface for
administrative tasks and viewing statistics and feedback. The web interface can also
be used for submitting exercises.
Fig. 1 High-level architecture of TMC
On the server side, the system is split into several components. On the front there
exists one or more web servers that are connected to a database, one or more version
control repositories (in our case Git) that contain the exercises, and a set of sandbox
servers. The database stores information of courses, exercises, users, submissions,
scores and snapshots. Each course is located in a separate version control repository,
where the exercises are added and updated by the course instructors. The sandbox
servers receive submitted exercises from the web server, run them on sandbox in-
stances (transient user-mode linux virtual machines), and submit the results back to
the web server. In order to handle high-demand situations (e.g. deadlines), TMC has
a queuing mechanism, and increasing the amount of sandbox instances is easy.