Darkened Software

Tag: Microsoft

Managers review duties do not just start at review time.

by on Sep.15, 2013, under Lessons

Review time can be a little nerve racking for employee’s at Microsoft given a rather large chunk of your yearly wages are based on it.  In addition to stocks and money the ranking also controls the possibilities for promotion and transfer during the next year.  While the employee creates the body of work that will be judged in the end, its is not even close to 100% in their control how they are going to be judged & ranked in the end.

The 3 well known factors at work here that cause employee’s to have so little control of their fate:

  1. You create objectives at the beginning of the year, you get to make some modifications at mid year if your manager approves.  But between then and the end of the year if the business objectives change you are likely going to be graded on something you have not been working on in 6 months.
  2. You are stack ranged against everyone else in the company within your level.  Its not enough that you were great in your engineering job but you have to be perceived as better at that than the same level producer, artist, designer, HR, sale, and marketing person is at their job.  Not sure there is a possible formula for figuring this out so as in any subjective process things are going to be off.
  3. As you get compare to a ever growing group of people in your band, you manager has to fight to keep their suggested ranking for you.  Then at each level after that the next manager up has to fight to keep you at your suggested ranking.  Thus it is crazy important that your manager not only has your back but is persuasive as well so that all managers in your line will also have your back.


Feeling a little nervous yet that all your hard work could be for not.  Don’t worry in reality you have even less control than that as there is one more huge variable at work.

4.  End of year only peer feedback.

At mid year you get a forced check in from your boss were you write up how you think your doing and he corrects you if needed.  This is great as it solves one big problem were people think they are going good but are really not and get surprised.  In this system someone now has 6 months to correct it before they get locked down for the year.

What this system does not prevent is both you and your boss thinking your doing great but at the end of the year when all the co-worker feedback comes in there is some discrepancies.  Now your completely screwed, if you had known earlier (mid year) you could have cleared up any real or perceived issues.  But since there is only 360 feedback at the final review, you find out just in time to realize your extra hard work was wasted.  This is not just a Microsoft issue, many companies use this non symmetric system.

One can try to protect themselves from these issues by asking their boss at every single one on one “how am I doing in general and am I on target to hit my end of year ranking goal?”.  Then everything gets corrected in two week intervals ( make sure your boss is showing up ) and if you do it consistently your mid year feedback will be only positive with hopefully plans on how to grow the next big skills set you need for the next band.

If your at the lead level or above you should already be talking to the other Art, Design, Production groups to find out if your team is meeting their teams needs.  This is a good time to find out feedback on how your individual team members are doing and if they are willing to share, how you are doing.  Not everyone is confident enough to give you feedback about yourself straight to your face, but I have never had anyone not tell me their thoughts about my employee’s are doing.  So there is really no excuse if you ever get review time feedback comments for your people that that you did not see coming.  If all the manager up the line are doing this then the system works out even if they will not give you your own feedback and thus no one gets surprised.

So there is 4 big things employee’s do not control that make their review an stressful nightmare of uncertainty.  The manager have the ability to control or minimize all of them.  It requires only a little bit more work above what you should be doing already, so there is no excuse for not to do it.  The payoff is massive as stress and uncertainty kill peoples productivity, and the trust gained from them knowing you have their back is massive.







Leave a Comment :, , , more...

2 weeks at Microsoft

by on Aug.18, 2010, under Development, Industry

10 years ago I was just was just finishing up a project with the 3DO company, not wanting to go through another dev cycle with I got an interview with Microsoft for the Direct X team.  At the time I was very big into openGL, but saw the potential in DirectX and wanted to be on the new leading edge of graphics.  They flew me out for the big interview but when I got there I found out that an internal transfer had already taken the position I was interviewing for.  At MS it is common they interview you for multiple positions and they still wanted to interview me for some Direct X Development Test spots.

Although not what I wanted after a full day of interviewing I had convinced them that I would be a great Development tester.  Slightly upset that the position I wanted was not available I stopped on the way back to the airport and got my hair dyed blue.  At the time I did not understand the rest of the benefit’s of working at Microsoft so I did not consider the offer letter enough to move out of CA and start doing testing again.

If I had known then what I know now I would have dropped everything and gone to work with them in an instant.  This company literally has everything a computer geek could want.

  1. You have access to more information than you could ever hope to learn.
    • ms library has access to every ebook and most tech and marketing books ever published.  You can check them out for 3 weeks and have 50 checked out at a time.
    • You have access to almost every consumer report and any marketing research ever done.
    • You have access to IEEE, AMC, and every knowledge database ever collected.
    • You have access to the people that originally wrote a lot of the the software you use.
    • They have more email lists, blogs, docs, meetings, round tables, video’s, industry committees and learning groups on any topic you could imagine.
  2. You have access to 30,000 very smart engineers that are willing to help you learn anything you wish, it is like the biggest college on earth.
  3. You have free or almost free access to all the software and hardware Microsoft produces.
  4. You have one of the best medical plans ever heard of.
  5. You have access to one of best fitness club in the world.
  6. You have access to the best legal team that you can also use for personal counsel.
  7. You have access to a billions of lines of source code from every project Microsoft has ever done.
  8. The amount of resources you have for a project is unreal.  If it gets backing it is almost unstoppable force.

So one of the great thing about going through a full interview is then you are in the system and the recruiters never loose contact with you. Anytime a position comes up that fits your resume they contact you right away.  Finally they called me up with a job I could not refuse.

In 2 weeks I have had more hardware resources dropped on me than I have in my entire career.  Been exposed to 2 different engines and more source code than I have seen in my entire life ( which is hard given how much time I spend on source forge ).  Checked out 2 dozen books from the library, downloaded Rosetta Stone for Japanese and finally had access to all game sales numbers imaginable for the first time.

I have gone from spending most of my time to trying to get resources to now trying to prioritize what tech / resources I want to learn / utilize next.  From trying to figure out what the publisher really wants to picking projects and features our team will help out with next.  From trying to just get access to information to trying to scope what information is most important to me so I do not get overwhelmed.

During orientation they discussed how the industry average for getting ramped up to full productivity was 3-4 months and at Microsoft people said it was much closer to 9 months.  Originally I laughed, but now I understood why.  You will reach your standard productive rate within your normal time line but here that is just the beginning as the bar is much higher.

Was recently talking to a college grad working there and it became clear he was squandering the mass opportunities he had lucked into.  Not sure you can ever fully appreciate it with having suffered in the under funded and disorganized world of normal software development.  Only then can one understand what it means to have everything you need and you are the only limit in what you get done.

Comments Off on 2 weeks at Microsoft :, , more...

Interviewing with two game console dev support teams

by on Jul.27, 2010, under Graphics, Programming

Some time ago I interviewed at Sony based on the recommendations of a friend who had joined their R&D department and was really loving his new job.  Their R&D department at the time only need a hard core graphics specialist but they had some other positions that they wanted to interview me for.  So I took a day and went to see what things were all about at Sony.  Most of their jobs were doing some sort of lead tech support for other developers so I expected I would get grilled on:

  • Tech & coding skills
  • Customer support
  • Documentation
  • Time management skills

After an entire day of interviews I had not had any questions on anything that was not tech & coding.  In fact many of the questions were so esoteric that I do not believe they were really looking for a programmer but a PS3 manual with a better personality.

eg. Interviewer:   How many instructions would this take on the cell processor

float temp = (bool) 0;

Me:  Not sure, why would you ever cast a bool to a float?

Interviewer:  You wouldn’t, but if you did how many instructions would it be?

Me:  Not sure, on the PS2 they had an Zero register so it would have been 1 instruction but since your asking I am sure that is no longer the case.

Interviewer:  See he does not know.

This in my mind clears up a lot of stuff, if they do not value customer support, documentation or coding samples to even ask about it in a interview then it is no wonder they are so badly know for the poorest developer support in the industry.

Contrast that to the interviews at Microsoft I recently went through for similar positions.  The first half of the day was all about tech and coding but they asked relevant questions about algorithms, error checking, comments and architecture.  Second part of the day was all about dealing with customers, creating white papers, writing good sample code and other tasks.  Last part of the interview was about what makes great games and user experiences, tech trends and how it will shape the games of the future and last how to communicate and let people know how to take advantage of it.

Perfectly clear why in one generation Microsoft has taken over the console market, they understand what is real dev support is and know that supporting the dev teams has to be the one of the core’s pillars in getting the best games.

Leave a Comment :, , , , more...

Improving gameplay testing in the console game industry

by on Apr.27, 2009, under Industry, Lessons

Seems like the average console game company’s gameplay testing process starts off with an email like this…

We’re having a playtest this Saturday afternoon, from 1:00 – 4:00. If any of you have gamer friends that would like to participate, have them fill out this NDA + survey and send it back to me.

People showing up are sat down in from of a dev box with the latest copy of the game and promised free pizza if they finish the game and fill out all the questionnaire.  Sometimes if the company is really on it, the testers play is recorded so it can be double checked and reviewed later by all the developers.  This testing is usually < 100 players and is typically only done only 3-4 times during the project at milestones like first playable, alpha, beta.   From this very small sampling of data feedback is given to the development team and at many publishers it is even used to help estimate pre-order numbers and determine a games slice of the marketing budget.

Its not that they do not get any useful data out of this at all, but nothing pulled out of this could be called qualitative and if someone tried to pass this off for testing in any other industry, they would likely be fired.

It is not the developers or publishers fault that there is little to no real focus testing, they do not really have a choice.  Console manufacturers only allow pre-release game disks to only run on special $10k development boxes that the public is not allowed to have.  So even though there are millions of xBox360’s out there, your testing pool is still only as many development boxes your studio plus publisher can allocate to it.

Even if your publisher has a ton of money to put in hardware, it is such a issue to coordinate getting that many people on site it can still only be done once in awhile.  The results of the game not being tested that often is so many changes are getting tested all at once that it is hard to determine or isolate what change is effecting the feedback.  There is also problems that we can not verify what live testers say about their playing habits or any other information they give us.

What has this done is placed undue burden on the game designers as they have to be able to come up with new and exciting game designs and get it exactly right as they will get very little chances for any real significant unbiased feedback before they ship.  With the punishment for unpopular levels being a pink slip is it any wonder that designers tend to go with tried and true designs that they are sure will work.  They might be more willing to take risks and move games foward if they had a quick feedback loop to tell them if they are heading in the right direction.

We need the kind of testing online PC games can pull off:

  • Easy distribution of game .
  • Connection to running games so info can be put in database.
  • Stats on user to ensure they fit our testing needs.
  • Online way to fill out survey and give feedback

We could have this kind of testing because the console manufacturers already have everything we need, we just do not have access to it.  They just need to let the developers get access to the play stats on xbox Live so we can pick the right testers for our games.  Then they need to allow us to release xBox arcade type pre-releases of the games so users can play test them for us.

It could work like this:

  • You create you’re xBox Live account and you select if you would be interested in getting access to games early for gameplay testing.  It warns them that their stats could then be searched by developers.
  • If you say yes you get put in a database and your playing stats of games is recorded so developers can decide if your good for the type of testing they need.
  • Developer can then send you messages that you have been approved for testing and requesting you download their game.
  • You Sign an online NDA and then start the game.
  • Once they quit the game it takes them to a survey page and they give feedback.

Suddenly with one database programmer and a community manager you could have access to and do coordinated testing with millions of users.

  • With the xbox Arcade type download system tests could get out to testers very quickly.
  • With access to that many testers and stats on them we could do things like control groups and very fine grain testing we have not been able to do before.
  • Given its all online your results are instant and developers can put in code to record all kinds of details you do not get from surveys or just video recorded sessions.  You could even save off a full replay of the session to get reviewed and analyzed later.

Potential issues:

  • Restrictions to ensure it is not abused for marketing purposes.
  • What to do during console transitions and your making a launch title as there are no consoles out yet.  Maybe that just becomes another hardship of being a launch title.
  • Developers will not longer be able to control screenshots and movies during the testing phase.  They can make them sign NDA’s but given that is it not done at their building things will end up online.

The potential gains in game quality and developer cost savings are huge as currently we often do not get feedback until it is way to late or too expensive to do anything about it.  I do not know if it would be as easy to implement as I think, but it can not hurt if everyone mentions it to their account rep.  Maybe someone at Microsoft or Sony will look into it.

2 Comments :, , , , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...