A victim of ones own success

Practical realities of adaptive software development

Stardock's business model has been predicated adapting to what users request.  The company's motto is "Innovation on demand".

The idea being that we release the best product we can and then we rapidly enhance it based on user feedback.  Users of Object Desktop, for instance, are very familiar with this.  We have also introduced this model to our game development which has had very positive results.  Our conclusion isn't so much that Stardock is "great" at customer service but rather most PC game companies are simply not geared up to do the kind of development model we're used to.

The PC utility market is extremely competitive and so rapid updates are one of our critical advantagers.  Since 1999, a huge percentage of Stardock's R&D budget has been put into building an infrastructure to crank out user requested updates quickly.

Central to that strategy has been Stardock Central which allows, literally, a developer to put up a build onto the system and have it available to users without creating an installer.  In addition, a custom set of forums that interact with the user's account information was created over the past few years that lets them interact with all our sites, Stardock Central, and access things quickly and easily depending on which site they're on.  It also has allowed us to implement various "community" features that helps make users part of the team -- because they are.

Where the model is starting to break down is as we've become more mainstream, the sheer numbers of users has started to show weaknesses in our system.  In addition, the demographic has dramatically changed which has significantly hampered our ability to do what we're best known for doing -- crank out rapid updates.

In the past, especially with Object Desktop, our demographic was almost completely all power users.  As a result, we could release alpha-level updates to those users who, in turn, would let us know what changes they wanted or any bugs they found. Power users who weren't comfortable with alphas would wait for betas and power users not comfortable with either would wait for releases.  Bugs, features, etc. would be queued up to be addressed and the system flowed nicely.

Over the past year or so, the utility market we're in, the desktop enhancement market has had two odd things happen: First, the market has increased in size dramatically. Secondly, the number of other providers of such software has strangely declined (despite more opportunity).  What happened to all these developers is subject for a different topic. But the net result is that WinCustomize.com, Stardock's site for supporting Windows customization, exploded with far more users than had been expected. Even now, it's basically limited by server levels.

The market size increasing though has brought in users who are not power users. Users who don't read manuals, don't understand the difference between alpha, beta, RC, release. Don't know how to resolve problems on their own.  The net result has been that the same updates that we used to just throw out onto Object Desktop (or ThinkDesk) now have to go through an internal QA or else we'll suffer in massive support calls (phone, email, more phone, more email) that drowns out truly serious support (In the last week of February, a sampling of tech support email resulted in the conclusion that roughly 2 out of 3 emails would have been resolved had the user simply looked at the readme in the given product or gone to the knowledge base -- two years ago, that would have been 1 out of 10 emails).

Of course, we're not set up with that kind of internal QA, not in terms of a long-term infrastructure.  When we do a major release like WindowBlinds 5 or Galactic Civilizations II we can put together a pretty impressive QA team using people from other parts of the company. But those people are far too expensive to have doing QA for the long term.  So now we're in the position of having to build a full time QA staff that can rapidly handle the updates that come from the various development teams and get them out there.

Similarly, we're going to have to start hiring people to do forum support. More people I should say.  In our rapid development model, we would put up an update that would list what's in there and users would respond with what they found worked/didn't work.  Now, we put up updates and people will ask "Did you fix <bug X>?" or "When is <Feature Y> going to be put in?"  That is, assuming the user even reads through the change log at all before asking questions.  Then there's the users who read the change log but don't try the update and complain about theoretical things.  "Oh, I don't like that new WindowFX feature, I think it might slow down my computer." or "I think the economic change in GalCiv II is bad" (without having tried it).

Of course, right now, we simply don't have those people to beef up a permanent large scale QA department or to act as a day to day buffer between development and support. They have to be hired and trained which we are, but it's a slow process.

So the business model of developing niche software (desktop enhancement utilities and turn based strategy games) in order to attract power users and hard core games has started to break down.  They have started to break down because those markets are no longer niche markets. Turn based strategy games have become mainstream and desktop enhancements are mainstream. And so we will have to evolve.  But it's been painful and likely to continue to be painful for awhile.

29,779 views 29 replies
Reply #1 Top
I sure can sympathize, even though I may not understand the finer details. Yet, you all have been doing an amazing job, especially in keeping us informed. Allowing us to see inside all that is developing is what makes this such a great community.

The only thing that bothered me is the way the article ended. Where is the light at the end of the tunnel? It had an air of 'things may get worse before they get better' to it. It seemed as though there was some sadness to the prospect that all of this is or has gone mainstream. I hope no mayyer what that the 'personal' touch that makes these sights so great, never diminishes. No matter what, I will stick it out as a lot of other people will. I'm patient and highly addicted to all these sights have to offer.

If I only had some talent to offer...and lived in Michigan.
Reply #2 Top
you've got a lot of support out there, brad - maybe you can rely on some more 'vollies' to monitor the forum while you bed the QA in? not me. i iz crap
Reply #3 Top
I'm betting your policy of listening to feedback and providing rapid updates also has the effect of creating more vocally demanding/critical customers by encouraging them to think about the product's flaws and raising expectations that they will be improved/fixed in the near future. That's a two-edged sword because if the fixes/changes don't come (or just not quick enough), those heightened expectations are deflated and you get upset customers who might have been otherwise content if they hadn't in the first place been encouraged to start thinking so much about what they'd like improved. But on the other hand, it's a real plus for many people that the developers are taking a really active role in post-release updating. Personally, I've bought at least 3-4 pieces of software I was on the fence on buying after seeing its developers in a forum interacting with customers and quicking releasing updates/fixes, so I'd say that the benefits of your rapid update model are worth the risks. Keep up the good work!
Reply #4 Top
I think your current business model is great. Incredibly demanding on yourselves though. I was wondering how you and your team could possibly keep up the pace since the release.

I personally don't play with beta's because I know I would not help in it's development.

Thanks for all your hard work. I cannot wait for next weeks release. It will almost be like getting a new game again. It will even have new features I have asked for, and what could be wrong with that?
Reply #5 Top
My sympathy. It suprised me how many people didn't know what a beta was. I thought any gamer should've known that.
Reply #6 Top
Welcome to the horror that is mainstream
Reply #7 Top
You may want to look at the totalwar.org as far as utilizing this growing community of users both as testers, and moderators.

in a recently released mod for RTW - the modders put the beta to the community which responded with zeal to both test, and document bugs, CTD's, and nice to have features - as far as I can tell its been working a treat

of coarse only a few community members are going to have the knowledge to do this effectively although if its structured well all users even simple ones (who dont read readmes) can easily find a place for constructive input, where it can then be filtered by more experience members of the community.

and as far as forums self moderation by senior and responsible members of the community who volenteer their time seems to work fine

evolution can be a good thing - a strong community of fans is an asset - utilizing this asset well will make your lives easier and make the community feel that have an integral role in the future of the game they love - in the case of totalwar it took the modders to do what CA could/would not
Reply #8 Top
but rather most PC game companies are simply not geared up to do the kind of development model we're used to.


Overly modest, it seems.
Reply #9 Top
That transition from samll to medium organisation can be one of the most vulnerable times in a company's life. But you have great products so I'd say the odds are in your favour.
Reply #10 Top
Ironically, the on demand/beta and release design process that doesn't work so well here is, more or less, the software engineering model Google uses and what Microsoft is trying to get into for Vista. As you've stated, the backend is most problematic, ie the Vale/Steam method of broadband software delivery. Best of luck to you and Stardock.
Reply #11 Top
I was for one was quite suprised to see post's like this one and others offering relatively rare insights into how a software devloper has progressed over the years. And its refreshing.

I think the real challenge for stardock will be (one that has probably all ready being addressed, if it hasnt been so all ready) "How do we not alienate existing 'hardcore gamers' or power users while at the same time appealing to wider interest, to further increase the market for our games", a more mainstream market. For example, talking about games here, If anyone compares Oblivion to the earlier Morrowind its not difficult to see not only the cosmetic changes (which were not interested in) but also changes to the fundamentals of how the title is represented to others. Quick example, the increasing abscence of statistics and accurate figures (abscence of anything other than a whole number in item stats for example) to appeal to a more mainstream audience. With the Elder scrolls series appearing jointly on console and PC for the first time ever, its something that will appeal more to casual console owners, rather than the more traditional 'hardcore' PC rpg players. Im sure many of these console owners would be far less PC savvy than a power user, so we can compare similar situations at stardock and Bethesda.

Even though it has been stated earlier that Stardock wont cater to the casual gamer specifically, with the kind of sales figures coming in, it would be silly to pass up on this oppitunity and not make a sequal to Gal Civ II that doesnt increasingly cater to the a more mainstream market of which Gal Civ II suprisingly found itself plunged into. But i fear in doing so, features that made the series popular with 'hardcore' gamers in the first place will be sacrified or dumned down.
Reply #12 Top
I'm actually very impressed with the direct relationship you all have with your customers and fans alike. Nevertheless, when loads increase, I guess it's not easy to keep up. Good luck with it anyhow, i'm sure you'll find bunch of trustable members of the community willing to lookout on forums for free!
Reply #13 Top

A victim of your own Success!

And yes, Users are ecpecting more immediate gratification and fixes.  I think this is the Microsoft syndrome where they got so blasted for security holes and were rushing them out to the tune of a couple a week. 10 years ago, you patiently waited for the next dot oh x release to patch your software.  Now, you want it patched today.

Reply #14 Top
If the wife would let me move to Michigan . . . . I'd be there to help formally.
Let me know when you open the Houston office.
Until then, I'll do what I can helping with support at WC.
Reply #15 Top
I would not recomend having volunteers do Forum Administration, unless you give them a very strict set of guidelines that are available both to forum users and the administrators themselves (obviously).

Reply #16 Top
Send me a collector's edition and a t-shirt and I'll be happy to answer questions on the forums.

Simple answer is to tier the access to updates. I read on the boards someone who was confused because a beta was available and didn' t know what a beta was or if they should be playing it. That kind of person shouldn't even have access to a beta!

You put a beta out there so it can get tested and you can get feedback. Make the beta group voluntary and talk about it only on the forums. If they sign up but ask dumb questions, remove them from the list.
Reply #17 Top
#16; You would want a 'stupid' person to have access to a beta because only a 'stupid' person can make 'stupid' mistakes that the application does not handle properly which makes it crash.

Assuming the 'stupid' person is able to document the steps taken, or even better, if the application itself tracked the users actions (Debugging, performance decreasing) and auto-launched a bug report when said crash/bug occured, you'd have a pretty good tester.

Course, then they would complain that their product failed...

So, yes, you want 'stupid' users to find the 'stupid' mistakes you've made in your application, but you dont want them to complain about them afterwards.



Reply #18 Top
It's the catch-22 of independent software houses. Due to your small size and nature, you are able to be more agile when it comes to reacting to customer needs. The result is a better product that blows the competition away. So you have a better product, you have better service and then the customers start to flood in (well, more of a gradually growing flow). Eventually you find yourself with a body of users too large to support in the way you have before, so you have to grow. But growth means you start to lose the agility of a small company if you don't do it very carefully. So it raises the question of "Is it actually a good thing to grow beyond a certain point?"

What's the point of diminishing returns? At what point does growing your customer base actually hurt the business you've worked so hard to build? Sure, growing may allow you to release more products, see greater revenue, but is service suffering? Do you lose the close contact you once had with your customers? Is it even feasable after a certain point to try and maintain that personal contact? Does it make sense for a company like Microsoft to have its executives and upper management getting involved on the community front-lines?

Popularity is a double-edged sword as you've said. It requires the company to grow, but growing too fast and too much could kill the culture and community that's been developed over the years.

I'm confident Stardock can handle this growth well, but I don't doubt the transition to a larger company will have it's challenges and growing pains.
Reply #19 Top
Brad -

I sympathize with the problems of your success, and as great as that success has been I hesitate to offer any advice, but you should spend a little time reviewing the app descriptions in SDC and looking back through those changelogs you say we don't pay enough attention to. The main description of ObjectBar in SDC still says it supports Win98 & WinME, for instance. I routinely check changelogs before running updates, but over the past couple of years, I'd estimate that there was no changelog information at all about a third of the time (the History link in SDC).

It's not surprising that people don't read those things when they've grown accustomed to them being so often out of date or absent. When the app documentation lags so significantly, which has been a "tradition" at Stardock, the casual users (as opposed to power users) easily get confused & frustrated by the "mismatches," despite the wealth of tutorial information available on WinCustomize. Even the software versioning system SD has adopted has been a point of contention for users, forever it seems. Why the tootip info and Help/About info can't match up with the actual version number is a mystery to the casual user. I don't consider myself a "power user" and my own preference would be to get the app now & documentation later, but that's just me - I suspect that devoting resources to being sure the descriptions & documentation are up-to-date on the front end will significantly cut down on the support demand on the back end.

Thanks for being so open about things and for soliciting user input in so many ways, and do accept this in the spirit intended. I want to see SD improve and succeed, too.
Reply #20 Top
lol, you don't have to give me anything and I'll still answer questions on the forums for you. I do that anyways...

But no matter how many I answer, it will never be enough. The problem that I see is how the forums are structured. The blog style interface works fairly well with the smaller community of hardcore followers, who are patient and knows how to work it, but when the user count goes over the roof, server load aside, the sheer amount of topics alone makes it nearly impossible to adress. So while you may answer one question, another one which adresses the exact same problem pops up a while later, because the first one was sufficiently answered and is now buried on page 8... Then, a few hours pass and another topic is created on the same problem, etc...


IMO, the first thing that really needs to be done is organization. I suggest splitting up the sections even more. I have never had problems when going into one of the smaller organized subforums, but on the main GalCiv2 forum, it is a complete mess. Hell, the sheer number of stickies alone could scare a normal person because it would take them days to read through. So here's what I think might help us, help you guys while you are looking for QA staff:

1. Break up the forums a bit further and don't list everything that is posted in the GalCiv2 section show up in the GalCiv2 category. Ideas goes into Ideas, and Modding goes into Modding only. This cuts down the number of topics that shows up and let people choose the types of topic they see, and help keep topics around. Basically adopt a more traditional "forum" style of posts goes into folders, etc... just for organization sake. I wonder how many people actually realize that you even HAVE subforums for ideas and suggestions...

2. Add a question and answer forum/section, with a stickied topic that is edited and maintained pointing to all the topics dealing with frequently asked problems. This should cut the number of posts down by half, if not more. Of course a QA person organizing it would help, but if not I'm sure someone can step up and handle it if you just sticky it for them. A stickied topic explaining how to properly use the forums will help immensely as well. I don't know how many times I've seen people complaining the quote feature is broken.

3. The ability to delete posts/comments. I don't know how many times I've seen people accidentally posted 3 times in a row because the system was slow to respond. This would really help make topics a bit cleaner. It would also reduce useless double topics. Add some moderators to clean the mess that people makes, and it will probably work.


About volunteers administration: I would say no, to a certain extent. This is mainly because I've seen some political bickering among forum administrators and that has never gone well. However, I'm not against a system that would promote knowlegeable and helpful people to a certain status, like "helper" so that when they answer a topic you can be safe to know that they are probably right. These people could be voted for by the community or chosen by the real moderators to help them out.


About betas: I would say don't post this to everyone in the GalCiv2 channels or journals or SDC. That would just lead to every average joe with no understanding of it trying it out. Instead, put up a post in an appropriately named "beta/in developement" section. You could sticky a post of the rules for using the beta as a disclaimer of sorts in the section. That way, only those that really wants it gets it. Tier access to beta updates seems a little extreme, and contrary to Stardock's fair treatment of their customers. But putting it into a "beta corner" and you will probably only have hardcore people playing with it.


Don't get me wrong, I like the fact that the forums can be accessed from all aspects of Stardock, it is an interesting little feature, but the shift to mainstream and the coresponding user base pretty much forces these kinds of changes. You pretty much need an in-your-face list of problems and frequenently asked questions/solutions to help out the new people and keep down the number of repeating topics and posts. Similarly, you also need to hide your betas away from those same people until they are ready. Organization is the key, I think you guys have a really great thing going, and I'm sure you'll come out on top of this situation.
Reply #22 Top
Trust me, if I had the credentials, i would want to work there myself.

I just did a presentation on GUI's thoughout history for school and will end up doing it agi n for my business workshop a few months from now. I sent a few (well, more than a few) people here already. Its just going to get bigger I guess for your guys as more poeple realize just what they can do wih their computer desktop and how they can intergrate online and offline with these technologies.
Reply #23 Top
My sympathy. It suprised me how many people didn't know what a beta was. I thought any gamer should've known that.


Haha, I totally agree. I read the GalCiv II forums very often, and its funny, but almost every time that Brad writes one of these interesting articles, I can pinpoint the exact forum topic that was the fodder for Brad's concerns. It may just be in my head, but there always seems to be a major forum topic that Brad comments in a day or two before he writes an article like this. Take the forum topic started in the past few days entitled "Do not download the second Beta 1.1!!!" in which the OP states there are a few problems he found and tells everyone else not to download the beta because it has problems. *rolls eyes* It would drive me mad! I really appreciate articles like this one written by Brad that give a little insight into how "users" like this affect the business of the company. Always an interesting read, Brad! Thanks!
Reply #24 Top
Yeah, good old mainstream. The further into it you get, the more users you get that put enormous support pressures on you. For a company like Stardock where two way communication between users and staff is large and visible, that can be incredibly dangerous.

To be honest I've been expecting this post for a few weeks. The masses of "I want THIS added to the game, why are they being evil and not changing the game for me?", "OMG I found a bug and reported it 5 seconds ago and it's not fixed, the game SUX", etc posts are quite depressing. Had the game come from Blizzard (not picking on them, they're just large and opaque) there'd have been none of that. With companies like Blizzard or Microsoft where NOBODY (Well there are exceptions. lol) expects to be able to actually speak with someone, mainstream users are far cheaper to support. Not expecting an instant reply, they might ask in an unofficial forum or even (GASP!) read the manual.

So what's the growing small guy to do? The important thing is to keep the power users happy and supplied with betas while keeping the support-heavy users at arm's length and away from betas. This is where 'level 1 support' can save far more money than they cost. Filter out the "is it turned on" type solutions that make it up 95% of mainstream support calls but more importantly give the power users a shortcut through to the existing systems as soon as they show themselves to be such.

Just make sure to get the balance right - keep the basic guy from getting confused and out of his depth without frustrating the genuinely knowledgable and helpful guy with "is it turned on? are you holding the mouse the right weay round?" questions.

Good luck!
Reply #25 Top
don't forget that you can also have *free* moderators (that is : they are not paid, they are just reliable fans willing to help)
most forums (even officials look at www.egosoft.com ) have moderators that are not here to answer questions but rather keep an eye on making the rules followed (such as : locking post answered in the faq or stickies, moving posts around in the right forum, merging threads talking about the same issue, deleting posts & threads such as 3 of the 4 posts of rosa here ...) and this allows the forum to continue working without too much problems.

anyway, I'm far from perfect but I wish you all the luck in the world, and to be able to handle all this new user base
keep up the good work