Classroom Software Evaluation

The question in front of them

We consulted with Radford University’s entrepreneurship and business program, including the MBA side, as they weighed a commercial business simulator for a graded place in the curriculum. The question came from program leadership rather than from one instructor choosing a tool for one section, which is what set the stakes: whatever they adopted, a whole cohort would be graded on it. Course software is a one-way door once a semester starts. Students are enrolled, assignments are built around it, and grades depend on it working. They wanted an outside read before that door closed, so they hired us to evaluate the software as software rather than as a sales demo.

We are not naming the product here. The evaluation covered a specific beta build from 2016, and it would not be fair to hang decade-old findings on a vendor whose product has moved on since. What travels is the method and what it turned up.

How we evaluated it

We used the platform the way a student in the course would have to. We incorporated a holding company, chose a jurisdiction, launched an operating company, sourced inventory, and ran the storefront: staffing, advertising, legal, and the paperwork underneath all of it. Then we came back across several sessions, because a simulator that runs on real-world clocks cannot be judged in one sitting.

Alongside that we tested the parts a course depends on but a demo never shows: whether the financial statements actually rendered (we checked in four browsers before calling it a defect), whether progress survived between sessions, whether the platform held up on the phones and tablets students actually carry, and whether an instructor could see enough of each student’s activity to grade it.

What we found

The first finding was a framing correction. It was being talked about as a game, and it is not one. It is a business simulator, and it models the administrative variables of running a company with real fidelity. That is a genuine strength for a course about running a company, but it sets a different expectation: students would not be pulled back to it the way a game pulls people back. That distinction changes how you assign it.

The onboarding front-loaded everything. A long tutorial arrived before any of it was useful, which is exactly how comprehension gets lost. Early decisions had consequences students could not see coming, too: starting with a retail company rather than a manufacturing one left us with nothing to buy on the open market and not enough capital to manufacture our own supply. That is a defensible lesson in a simulation, but only if the instructor knows to expect it. Pacing compounded it: inventory took a day or two of real time to arrive, and there is nothing to do in the meantime. The simulator wants short, frequent check-ins across a semester, not a class period. Its availability on phones, tablets, and desktops was the right answer to that rhythm, and it was one of the clearest things the development team had gotten right.

Then there was the beta risk, which was the part that mattered most for a graded course. Screens that were present but inert. Profit and loss statements that rendered their structure and none of their figures, for either company, in every browser we tried. And after several hours of play, a session that came back to the tutorial screen with the work gone. Account access was throttled during the testing period as well, to the point that sign-up was closed on one of the days we tried to use it.

What we recommended

Our recommendation was conditional rather than yes or no, because that is what the evidence supported. The simulator was worth using for what it taught, and the conditions for using it safely were specific: settle the number of student seats with the vendor before committing it to the curriculum, capture student progress on a regular cadence so a data loss does not become a grading dispute, and set expectations with students up front about pacing and about the early choices that quietly constrain everything after them.

On grading, the news was better than expected. The platform generated a per-student progress page automatically. The metrics were thin, but they were enough to see how a student was actually doing, which was the specific worry they had raised.

Beyond the simulator

The software was the concrete question, but it sat inside a larger one the faculty were already asking: how to grow the entrepreneurship program itself. The engagement extended into that. Alongside the evaluation we gave them a framework for thinking about that growth and named the factors we believed would matter most, so the simulator decision got made inside a view of where the program was heading rather than on its own.

Radford’s read afterward was that the work gave them an “excellent framework” and “identified some key factors” for growing the entrepreneurship program. That is the part that outlasted the software. A verdict on one product is worth a semester. A way of deciding what belongs in a program, and why, is worth every tool decision that comes after it.

Why this one is still in the portfolio

This is the oldest consulting engagement we still show, and it is here because the shape of it never changed. Someone is about to commit an institution to a piece of software, and they need an honest read from someone who will use it hard, name what breaks, and say plainly what would have to be true for it to work. The subject has moved from classroom simulators to enterprise platforms and AI assistants, and the job is the same one.