18-649 Project Assignment #10

Due Thursday April 3, 2008 11:59 PM

Rest of Implementation; Unit Tests; Some Integration Tests

Please submit all project-related correspondence to ece649-staff "at" ece.cmu.edu

In project 8 you designed a more advanced dispatcher to efficiently deliver passenger loads. In Project 9 you completed this design by creating unit tests to exercise these control modules, along with the implementation and testing of your new dispatcher.

Now in Project 10 you are going to finish implementing the rest of your design, write at least 3 comprehensive integration tests, and actually execute the tests you previously wrote and new passenger workloads. Instead of simply running tests, finding bugs and fixing them, we will ask you to create a test log. This log will allow you and us to quickly see what tests you have run, their outcome and any changes you had to make. A test log serves as a simple tool to measure the coverage of your tests and to highlight areas of your design where there were problems.

Assignment:

Project 10 requires you to write at least 3 comprehensive integration tests, implement (write the code for) the remainder of your design, execute the tests you wrote last week (in Project 9), and log their results and submit the files which have updated and changed as a result of faults found. You can of course create and execute additional tests even beyond what was in Project 9 if you wish.

  1. Finish implementing in Java your improved elevator system. Please follow the coding guidelines from previous projects. Be sure to include traceability information (e.g., mark transitions in comments). This means get all the code done for the entire new elevator functionality.
  2. Execute all the unit tests you developed in Project 9. Don't wait until the last minute -- many people find that their tests themselves have bugs in them that need to be worked out before they will even run, and only then can they find and correct bugs in the code they are testing. We don't expect you to fix all the bugs in the elevator this week if you find a lot of them, but we do expect you to get your tests running and tell us what bugs you'll be fixing the following week.
  3. Write a minimum of 3 new integration tests for your new design (more is better, but this is the minimum). These new tests should differ noticibly from tests you've previously created or that we've previously provided. The idea is to get a bit more coverage to get you ramped up for next week, when you'll need to do thorough integration testing. Execute these tests. Again, the point is to get the tests running and report results. Fix elevator bugs if you can, but if you can't that's OK as long as they get fixed next week. We strongly suggest you run all the integration tests you already have developed, but running the three new ones is the bare minimum.
  4. Run the four new passenger tests below and report what happened. If there are bugs then just say as much as you can figure out in time available. We don't expect everything to really work this week, but it's important to get a sense of how you're doing as you go into the test & debug phase for next week.

Important overall project goal for this week -- get all the tests running and tell us how it turned out. Fixing elevator bugs should come after getting all the tests in place. Do as much as you can, but getting pretty good code in place and creating and debugging the tests themselves is the emphasis this week. No points off for elevator bugs as long as you are honest about your test results.

Important note -- as some of you have noticed already, figuring out whether the elevator is really working in every respect isn't necessarily easy with just a GUI. Next week we're going to ask you to create some sort of run-time monitoring (either on line or using log file analysis) to make sure you are meeting all the elevator high level requirements. This is a good time to get started, but that won't be a hand-in until next week.

We are providing the following new passenger tests files:

Log Files:

Please include all the results of executing your tests in files with the name having the form of #TestName#.results where #TestName# is the name of the test being run. Put the results in the same directory as the corresponding tests they apply to.

Here are requirements for the test logs:

Create one test log file in tabular form for each phase of test (unit, sequence diagram, passenger) documenting several aspects of the test. An example table is presented here as an Excel file. Your final turn-in of tables must be in PDF format, and fit on one page wide at most (landscape is OK -- but they have to print one wide with legible font no smaller than 10 point). This means that your table must not be wider than the page and require 'stitching' together. Put all these files in the logs subdirectory and provide pointers from your main project portfolio page

For Unit Testing, your table should have columns for each of the following:

  1. Name of test file being run.
  2. Object or statechart being tested
  3. Special instructions or conditions for this test. This helps us understand how you performed your test, but also ensures repeatability for you. You must provide clear instructions, we will run your tests and must be able to reproduce your results.
  4. Expected outcome of the test (pass condition).
  5. Initial Test outcome ('pass'/'fail').
  6. If your test failed, what transition or state your test failed in.
  7. If your test failed, state the source of the failure (the test file itself, or the control): give the file name, line number, and trace the source of the failure. This must also be recorded in your defect tracking. If unknown say so, but you'll need to figure it out by next week.
  8. Final Test outcome ('pass'/'fail').

For Integration Testing, your table should contain:

  1. Name of test file being run.
  2. SD being tested.
  3. Special instructions or conditions for this test. This helps us understand how you performed your test, but also ensures repeatability for you. You must provide clear instructions, we will run your tests and must be able to reproduce your results.
  4. Expected outcome of the test (pass condition).
  5. Initial Test outcome ('pass'/'fail').
  6. If your test failed, what arc(s) your test failed at.
  7. If your test failed, list the source of the failure (the test file itself, or the control): give the file name, line number, and trace the source of the failure. This must also be recorded in your defect tracking. If unknown say so, but you'll need to figure it out by next week.
  8. Final Test outcome ('pass'/'fail').

For Acceptance Tests (passenger tests), your table should contain:

  1. Name of passenger test file run.
  2. Brief description of passenger load (deduced from .pass file), such as, 'heavy up-peak traffic, over 20 passengers in 2 minutes.' This will allow you to see how your elevator performs under many situations.
  3. Initial Test outcome ('all passengers delivered'/'passengers delivered, unsatisfactory performance', 'passengers not delivered'). At this stage you can begin to tune your elevator for passenger workloads. If you are satisfied with your elevator performance now, know that you will have to improve it later!
  4. Simulated minutes and seconds required to deliver all passengers (i.e., time elapsed in terms of passenger's view of time, not time it took you to run the simulation in elapsed "wall time").
  5. If you failed to deliver passengers or improve performance: trace to the source of the problem, including file names and line numbers, if known. You must deliver all passengers! Remember defect tracking! Again, if you don't figure this out this week, OK, but you'll need to provide it later.
  6. Final Test outcome ('all passengers delivered'/'passengers delivered, unsatisfactory performance', 'passengers not delivered')

When you fail a test you must eventually correct the problem and be able to produce a pass -- but that can wait until next week if you have tricky problems. We encourage you to fix as many bugs as possible, but don't expect perfect code. Please remember to update your defect tracking in response to your test failures!

Design portfolio

Your portfolio should be up to date and reflect all changes through this project.

Hand-in requirements are the same as for previous projects, with any obvious extensions to accommodate the new items in this project phase.

Grading (75 points):

Grading will be as follows:

Note: above points include relevant updates to traceability, change log, defect tracking, etc.

Note: we are no longer awarding points for having a complete and consistent design package, nor just for following instructions. Those should be a given at this point. However, TAs can deduct points from the project for failing to following turn-in directions or having an end-to-end design package with blatant gaps.

Back to course home page