18-649 Project Assignment #11

100% Integration Test Coverage; Instrumentation; Acceptance Tests

Assignment:

There are four main parts to this assignment:

• Unit Tests
    1) You must rerun all unit tests from previous project that demonstrated problems. Fix the problems, and rerun the tests.
    2) Rerun all unit tests to ensure that they all pass and no new problems have emerged.
   
• Integration Tests
    1) You need to have 100% coverage of all SDs for every object. Therefore you need to add integrations tests until all SDs are covered.
    2) You need to run all new integration tests, if problems are found, fix them and rerun integration test.
    3) Run all unit tests and integration tests to make sure that no new problems have emerged.
    
• Acceptance Tests
    1)  You must rerun all acceptance tests from previous project that demonstrated problems. Fix the problems, and rerun the tests.
    2)  You must create at least 4 passenger load files: (1) heavy up-peak traffic, (2) heavy down-peak traffic (3) heavy middle-peak traffic, (4) and a test that has two heavy loads with a long break in the middle with no load. You must run these tests, if problems are found, fix them and rerun tests.
    3) Run all unit tests, integration tests and acceptance tests to make sure that no new problems have emerged.
    4) You must demonstrate that your elevator system meets the new requirements given in Project 8. To do so you must come up with an novel method for testing your elevator and be able to demonstrate this. You must document your test method and provide a clear write-up describing your testing methodology. You will not be able to observe the GUI to show that you meet requirements. In general you will likely have to add instrumentation to gather data, and then process this data. Your testing methodology must clearly demonstrate how the high-level requirements are met. If there is a violation of these high-level requirements, the violation must be traceable to the source. You must turn in all modified code for instrumentation, processing, etc. More specifically, you must create a write-up that describes your instrumentation approach. This write-up should be approximately one page. You must demonstrate that your testing scheme catches all 5 requirement violations.
   5) Run all acceptance tests using the you new method described in 4) above. Fix all problems found and rerun rests.
   
• Re-run all unit tests, integration tests and acceptance tests to make sure that no new problems have emerged.
   

Please include all the results of your unit and sequence diagram tests in the form of #TestName#.results where #TestName# is the name of the test being run. You are required to record these actions in your test log:

For Unit Testing, your table should have columns for each of the following:

1. Name of test file being run.
2. Object or statechart being tested
3. Special instructions or conditions for this test. This helps us understand how you performed your test, but also ensures repeatability for you. You must provide clear instructions, we will run your tests and must be able to reproduce your results.
4. Expected outcome of the test (pass condition).
5. Initial Test outcome ('pass'/'fail').
6. If your test failed, what transition or state your test failed in.
7. If your test failed, state the source of the failure (the test file itself, or the control): give the file name, line number, and trace the source of the failure. This must also be recorded in your defect tracking.
8. If your test failed, you must correct the problem in your code, and rerun the test until it passes. You must also document the solution.
9. Final Test outcome ('pass'/'fail').

For Integration Testing, your table should contain:

1. Name of test file being run.
2. SD being tested.
3. Special instructions or conditions for this test. This helps us understand how you performed your test, but also ensures repeatability for you. You must provide clear instructions, we will run your tests and must be able to reproduce your results.
4. Expected outcome of the test (pass condition).
5. Initial Test outcome ('pass'/'fail').
6. If your test failed, what arc(s) your test failed at.
7. If your test failed, list the source of the failure (the test file itself, or the control): give the file name, line number, and trace the source of the failure. This must also be recorded in your defect tracking.
8. If your test failed, you must correct the problem in your code, and rerun the test until it passes. You must also document the solution.
9. Final Test outcome ('pass'/'fail').

For Acceptance Tests (passenger tests), your table should contain:

1. Name of passenger test file run.
2. Brief description of passenger load (deduced from .pass file), such as, 'heavy up-peak traffic, over 20 passengers in 2 minutes.' This will allow you to see how your elevator performs under many situations.
3. Initial Test outcome ('all passengers delivered'/'passengers delivered, unsatisfactory performance', 'passengers not delivered'). At this stage you can begin to tune your elevator for passenger workloads.
4. Simulated minutes and seconds required to deliver all passengers (i.e., time elapsed in terms of passenger's view of time, not time it took you to run the simulation in elapsed "wall time").
5. If you failed to deliver passengers or improve performance: trace to the source of the problem, including file names and line numbers, if known. You must deliver all passengers! Remember defect tracking!
6. If your test failed, you must correct the problem in your code, and rerun the test until it passes. You must also document the solution.
7. Final Test outcome ('all passengers delivered'/'passengers delivered, unsatisfactory performance', 'passengers not delivered')

Please remember to update your defect tracking in response to your test failures as well as document the solution!

Design portfolio

Your portfolio should be up to date and reflect all changes through this project.

Hand-in requirements are the same as for previous projects, with any obvious extensions to accommodate the new items in this project phase.

Grading (90 points)

• 10 points for unit testing
• 20 points for integration testing
• 20 points for acceptance/passenger testing
• 20 points for instrumentation testing scheme and write-up
• 15 points for demonstrating testing scheme catches all 5 violations of high-level requirements
• 5 points: what can be improved about this project? Include even minor bugs so we know to fix them. If you found nothing, then so state.

Note: above points include relevant updates to traceability, change log, defect tracking, etc.

Note: we are no longer awarding points for having a complete and consistent design package, nor just for following instructions. Those should be a given at this point. However, TAs can deduct points from the project for failing to following turn-in directions or having an end-to-end design package with blatant gaps.

Back to Course home page