Survey Questions: Organizational Self-Assessment

With the advice and consent of the management, I’m trying to gather quantifiable data that will guide our future knowledge-transfer efforts. Thank you for participating.

Current Project / Near-Future Needs:

The first questions are about your personal skills and contributions to the near future of your current project. If you lead a team, that will be on a separate screen.

1. What tool or technology would be most valuable for you to brush up on?
2. What aspect of the development life-cycle would be most valuable for you to learn more about?

Current Project / Attitudes and Impressions:

These next questions relate to your current project. If you’ve recently changed projects, please answer with respect to the project you’re currently most familiar with. I will not ask what project you’re on. (I fear it might poison the data.)

To keep things simple, I’ll ask for letter grades on each of several aspects of project execution:

• A: Excellent; exemplary.
• B: Good and respectable.
• C: Passable, but would benefit from improvement.
• D: Needs improvement badly.
• F: Fail.
• K: I am not well-enough skilled in this topic to evaluate our performance.
• N: I have no visibility into the team’s performance on this topic.

---

• Mission and Vision: Does everyone know what makes this project special and worthwhile?
• Systems Thinking: How well do you understand the project’s role within, and interface to, the larger mission and context beyond the project? Thank you Margaret Hamilton.
• The interplay between software and engineering through such topics as the ACM code of professional ethics or the development of engineering ethics as a phenomenon (which is historically recent).
• Planning and Roadmap: Thinking up and evaluating possible future features or lines of development.
• Release Cadence: Do meaningful increments of business value go live regularly, or do technical obstacles slow the pace?
• Reliability: Does your product or service just work, or do you spend much time dealing with mistakes and breakdowns?
• Feasibility analysis: The economic sniff-test for each new concept, feature, or requirement. Accounting for the costs and high-risk hurdles that might stymie project completion, balancing these against viable mitigations, and roughly estimating the likely ROI on implementing.
• Procurement: Obtaining physical and logical resources on which to develop, test, or deploy the solution, service, or product.
• Deployment: Connecting, configuring, and installing the software on computing machines so that stuff physically works.
• Configuration Management: Keeping track of what versions of which artifacts are meant to be part of which release. Also, version control.
• Requirements / General: Is there an up-to-date, detailed, and well-organized record of what the product is meant to do and how it’s meant to work, such that someone unfamiliar could efficiently come up to speed on any arcane aspect? Or do you instead rely on individuals and their falible recollections to be fonts of subject matter expertise, subjecting your project thereby to the hazard of someone leaving or getting hit by a truck?
• Requirements / Traceability: Do you have a clear and detailed picture of why each bit of your application exists and behaves as it does, or do you frequently wonder whether some oddity is a bug, a feature, or a vestigial relic?
• Architecture: The abstract plan to divide and conquer the overarching problem. Modular breakdown and overall approach to design.
• Design / General: Concrete and understandable solutions to specific, detailed subproblems: are we doing that well?
• Design / Plans and Records: For nontrivial problems, you probably sketched out some design notes in the process of convincing your team that an approach would work. Can we find those sketches or notes later when someone has a question about said design, or do we throw it away and have to figure it all out again when the time comes?
• Code / Organization: How easy or hard is it for a new person to find the modules they need to read or update?
• Code / Dragons: Give a bad grade if there are large sections of code that nobody knows how or why it works.
• Code / Simplicity: Is your project’s code trivially-easy to read, understand, and modify even at a deep level, or does it contain lots of long, inscrutable procedures where people are afraid even to look (much less touch anything) for fear of breaking things?
• Code / Extremes: How would you grade the single worst function or module in your application, that you know about?
• Inter-Team Negotiation and Persuasion: For example, say your project has some new feature under development, which will also require Team XYZ to accomplish a task they might not otherwise do any time soon. How hard is it to achieve and recognize alignment on schedule and expectations between your combined sets of stakeholders?
• Production/Operations Management: In our context, I mean planning, control, execution, and administrative accountability for capacity, monitoring, support, and incident response.
• End-User Documentation: Can a person unfamiliar with the specifics of your product (a) figure out what to even look for, and (b) find it?
• End-User Support / Availabillity: Do people trivially find the help they need when they need it, or is it a long slog to find your inadequate homepage or contact list? Do new customers have to guess at how to properly request help or file a defect report?
• End-User Support / Boundaries: Do you clearly publish the boundaries on your scope of support? Have your customers understoood and agreed to these bounds? Does the support staff know how to say “no” when it’s called for?
• End-User Support / Transparency: When you have a support interaction, do customers know what to expect and when, or does the squeaky wheel get the grease?
• End-User Support / Sustainability: Does each new customer increase your own burden? What about the overall collective burden imposed on the company stemming from that relationship?
• Operations Support: When something does go wrong, how quick and easy is it for trained personnel to notice, identify the cause, and fix the problem?
• Quality Assurance: Code test coverage is not enough. How do you know that the system does exactly what it’s supposed to, even in weird rare corner cases?
• Team member professional development: Are you getting better at this job, fast enough?
• Project Control: Track, report, and analyze the plan, forecast, budget, schedule, scope, progress, effort, process, and changes to the above.
• Estimating: Business would like to know how much time, money, and effort things will take – for business reasons. Do we consistently make, report, and improve these estimates on an objectively data-driven basis, unmoved by desire? And are they properly understood to be estimates (with a margin of uncertainty), not commmitments?
• Project Management: Sets goals/priorities, manages changes to scope/schedule/budget, allocates resources, and facilitates success. Primary focus is mission accomplishment on time, under budget, and with a satisfied customer.
• Product Management: Lives at the intersection of Business, UX, and Tech to invent, guide, and evolve the product. Primary focus is the grand vision and outside stakeholders.
• Intra-Team Communications: quantity, quality, tone, dignity, completeness, clarity, and frequency.
• Extra-Team Communications: quantity, quality, tone, dignity, completeness, clarity, and frequency.
• Overall Process Maturity: For what fraction of our activities do we consistently follow a defined process? Do we periodically assess and evolve, or do we shoot from the hip?
• Stress Level / Vibe: Do you work in a calm, motivating, and professional environment with clear sequencing, or do you fight fires in a culture of constant urgency where the loudest voice wins?

Demographics:

Regardless of rank or pay bracket, on your project are you:

• Team Lead
• Senior Contributor
• Junior Contributor
• Other

Questions Specifically for Seniors:

If you are a Team Lead or Senior Contributor:

1. What do your more junior colleagues most need to improve on? (Free-form)

2. Why is this?

3. On the same grading scale as before, how do you grade the overall code quality on your project? Do you stay tightly factored, with functional cohesion and low coupling, or do you have bad copy-and-mutate with interlocking temporal dependencies? Can you understand your code declaratively from denotational semantics, or does it require tricky operational reasoning?

4. Are the more junior people learning as much, and as quickly, as you’d hoped on this project? Not just about the project itself, but overall software engineering skill.

Write-In Section:

• Is there anything else you’d like to share?
• Do you have any questions of your own, that you’d like to see asked?

Finally:

Thank you!