CS4500 Verification and Validation Results 3

April 11, 2008

Team Here For Beer



1. Introduction and Overview

1.1 Purpose of this Document

The purpose of this document is to describe the results of the verification and validation performed as described in Verification and Validation Plan document.

1.2 Scope of the development project

A real-time strategy game titled: Here For Beer Tower Defense will be developed. The game will be made using C++ and the Microsoft DirectX 9.0 SDK. The game will be similar in genre to current hit Flash-based games (see: 1.4 References Section) but will harness the power of modern graphics hardware. The single player game will consist of a single grid space in which a number of "maps" (number TBD) can appear. Each map will define a path in which the waves of enemies during that game will progress. The player can stop the enemies by building and upgrading towers on the grid, which will automatically attack enemies that are in their fire distance radius (in the tradition of typical tower based real-time strategy games).

The game will provide lasting single player value through its dynamic action and large spectrum of strategy, rather than through large amounts of graphical assets.

The project will also include a web front end programmed in Adobe Flash 9.0. The single player game can upload high scores via the internet to our server, which can later be browsed via the Flash-based front end.

1.3 Definitions, Acronyms, and Abbreviations

  • DX9 - DirectX 9
  • HFBTD - Here For Beer Tower Defense
  • RTS - Real-Time Strategy
  • SRS - Software Requirements Specification
  • Twitch Game - A type of game where users must react quickly to circumstances in order to continue playing (such as Tetris)
  • UI - User Interface
  • Wave - A Group of enemies, or other entities that will arrive, and attempt to depart together.
  • VVP - Verification and Validation Planning

1.4 References

Flash Tower Defense Game, Vector Based
Another Tower Defense Game

1.5 Overview of Document

The Verification and Validation Report is a document outlining the testing, structure, and verification of the software that has been completed.

2. Summary of Results

All tests complete up to this point have been successful. Some issues were discovered as a result of the process described in the VVP (mainly through user testing). These issues have all been dealt with and resolved. For example:

  • We have added made subtle adjustments to some of our long standing algorithms such as particle system and tower placement system. All of our tests still pass.
  • We have added in a rudimentary level system, so far all tests related to this have passed.

3. Results From Reviews, Walkthroughs, Inspections, and Audits

Our last weekly meeting was very helpful to get everyone into the same frame of mind for the final 2 weeks before deadline. Through code review we were also able to resolve and tweak a few sections of the game to make them perform more much better.

Our Quality Assurance tests consisted largely of Code Viewing, and Unit Test writing. From these reviews, the reviewer requested additional Walkthroughs, Inspections, or Audits as necessary.

  • Additional walkthroughs were performed on several components of the system.
    • Path finding
      • Map path structure
    • Particles
    • Primitives
  • Our two tier test plan and procedure results:
  • Test First procedure Results:
    • Each team member has done a great job in first testing their components before completing an assigned task as part of their regular development process. Each source code commit has been thoroughly tested by the programmer before committing.
  • Weekly Quality Assurance Reviews Results:
    • Weekly reviews were conducted during our weekly meetings. They provided great feedback to the developer and gave the developer a chance to teach the team how that particular component works with the rest of the system. In some cases recommendations were made for changes to be completed the following week; in other cases, everything worked as expected.

In as far as the implementation has gone, few unit tests have been written. Most tests and quality assurance performed were via the methods described above.

4. Results from Testing

4.1 Summary of Component Test

The team has been very good about keeping the project clean. We have no incidents in where a team member broke the code, preventing other members from completing their task(s).

Component Pass/Fail Explanation Things Left To Do
Graphics Rendering PASS Particle effects have been significantly enhanced, all tests continue to pass. Audit Z-rendering.
User Interface PASS Additional missing components of the user interface have been added. The interface is now near feature complete. Make interface more eye-pleasing
Gameplay System PASS There is still a lot of balance work to do, but the current game is very playable and spans multiple levels Add additional enemies, implement score
High Score Server PASS This is one of the last components the team will be focusing on. The score submission code has been written and tested. The web server script has been written but as of yet has no front end. The web server script is passing all tests. Write flash front end.

5. Evaluation of the Process

In general, our process has worked very well. We've been able to keep bugs to a minimum.

5.1 Evaluation of Test Cases

The cornerstone of our process is the weekly meetings. In these meetings we have done most of our review work, discovered issues, resolved issues and assigned new reviews to be completed by the next weekly meeting. While few unit tests have been written, they have proved effective at preventing regression. Also, visual inspection by our build master Dax has allowed him to spot many issues. This has resulted in a cleaner, more well designed product. Other team members have done a great job of finding problems and recommending solutions and design philosophies to improve the system.

5.2 Results from Defect Tracker

The weekly meetings allow us to assign tasks to developers. These tasks are written down in our weekly reports, which serves as our defect/task tracker. At our weekly meeting we review the list of tasks.

5.3 Lessons Learned

Our process has worked very well for us so far. More unit tests will be required as we move to less visual parts of the system. So far the whole team agrees that everything has gone very smoothly. We feel that we are making good progress and have worked very hard to get to where we are now. There is still plenty of work to do and we hope that we will complete all the components and tests we have described in these documents.

6. Outcome of Acceptance Test and Delivery

6.1 Acceptance Test Summary

Systems Tested

  1. Graphics Rendering - PASS
  2. Gameplay Logic - near-complete, needs polish
    • Map System - PASS
    • Map Pathing - PASS
    • Level System - incomplete
  3. Highscore Uploading - PASS
  4. Highscore browsing - incomplete
  • Testing Schedule and Resources: System tests for all implemented systems have passed.
  • Test Recording Procedures: Tests for rendered graphic have been kept in order to continually confirm that no component implemented has broken the functionality of another system component.
  • Hardware and Software Requirements: The system environments have provided few obstacles. Each team member been able to run the application and has a machine capable of meeting the hardware and software requirements.
  • Constraints: Testing has been limited to methods described above. Extensive tests are beyond the means of this project.

6.2 Graphics Rendering and User Interface Testing Results

The game play has remained very smooth and several optimizations have insured that we stay with in our hardware specs for an enjoyable gameplay experience. Particle effects and animations are in place and working.

  • Input: User Interface Interactions - PASS
  • Expected Output: Known responses to User Input - PASS (though still somewhat incomplete)
  • Expected Output: Graphical environment - PASS

6.3 Gameplay Logic Testing

This entire component is still in progess, players are now able to select maps at game startup and the level system has been added. It still needs balance and further tweaks.

  • Input: User Interface Interactions - PASS
  • Expected Output: Control of graphical environment - PASS
  • Expected Output: Enemy interacting with towers - PASS

6.4 High Score Server System Testing

The server has been written but the frontend is incomplete.

  • Input: High Scores from HFBTD game client - PASS
  • Expected Output: List of all High Scores from HFBTD High Score submissions in XML format - PASS

7. Summary of Open Issues

We are getting ready to enter the final stages of game development. The game is currently near feature complete. After the few remaining tasks are completed we will spend the rest of the development time enhancing and balancing the gameplay logic.

8. Additional information