18-649 Project 12

Check the course webpage for due dates

Please submit all project-related correspondence to


In this project, you will finish your runtime monitoring, complete unit and integration testing, and begin acceptance testing.


1.  Acceptance Stress Testing

1.1  Run Some More Acceptance Tests

We've provided a few more acceptance tests for you. You should look at these tests as good examples of tests that your generator (in part 1.2) might generate. You should run these tests and update your Acceptance Test Log. This includes linking the test files, and filling a row in the table with the test statistics. You must also update the test results for the previous project acceptance tests with the results from your new elevator design.

Any tests that you do not pas should have a justification in the notes section of your Acceptance Test Log that describes the bug causing this behavior. This justification should include the number of the bug in the Issue Log (create a new entry in the log if you need to).

1.2  Write an Acceptance Generator

For this project you will need to write an acceptance test generator. Your acceptance test generator: Since your test generator will use some form of random number generator, it's okay if not every generated test satisfies the conditions, however your generator must be able to create tests that satisfy this criteria. This means that if your test generator only sends passengers from 1 FRONT to 2 BACK, this is not an acceptable generator.

Your acceptance test generator can be written in whatever programming language you feel like, however, it must run on the ece machines. Additonally, if the generator is in a compiled language, you must include the source, and an executable that will run on the ece machines. For this reason we highly suggest a scripting language, such as Bash, Perl, or Python.

Here is a template of a bash script that will generate some tests that do not meet the criterion. Include this test generator in your /acceptance_test folder.

1.3  Create and Run 40 Tests

Using your test generator, create and run 40 tests. Link them in your Acceptance Test Log in a separate table below the first. Use this table for stress tests, and the original table for the acceptance tests that have been provided (e.g. proj7acceptance1.pass) or that you have created yourself.

2. Write a Monitor

Write runtime requirements monitors for RT-8.1, RT-8.2, and RT-8.3, and add them to the RuntimeRequirementsMonitor java file. Information about runtime monitoring can be found in Project 8. If there is a group member who has not yet written a monitor, they must write one for this project. If you are in a group of three, and everyone has written a monitor, then this task can be assigned to any group members.

We highly advise that you run your monitors on your mid-semester elevator (Sabbath mode), to make sure your monitors do in fact catch violations of the high level requirements. If your project 7 elevator does meet any of the high level requirements, we advise that you introduce behaviors that cause violations, so as to ensure your monitor is not generating false negatives.

For the final demos, you will be required to pass your acceptance tests with no warnings generated by the course monitor. The course monitor is quite slack, but you should still make sure your monitors are actually detecting faults, to give yourself plenty of time to debug your code prior to demos.

3.  More Testing

3.1  Complete Unit Testing

Run all unit tests.  For this project, all unit tests must pass (0 failed assertions). 
You should have already passed all of your unit test for last project, so this should be a freebie.

3.2  Complete Integration Tests

First, double check your sequence diagrams to requirements traceability to make sure it is up-to-date.  Make sure your sequence diagrams are consistent with the design changes introduced in project 8 (and any updates you made in project 9 and 10), as well as consistent with your updated message dictionary from project 11.

If needed, review the Integration Testing section of the Testing Requirements document. 

Once your sequence diagrams are fully up-to-date, you will finish updating your integration tests, completing the remaining integration tests that you didn't update last project. If you have been keeping up with your issue log, it should be easy for you to use it to identify which integration tests need to be created or updated.  You should also make sure that the the traceability comments in all of your integration tests are complete and up-to-date. 

Update your Integration Test Summary File with the new integration tests.  Use the verification script to ensure that the summary file is correct and complete.

When you have completed this part of the project, you should have at least one integration test for every sequence diagram.  Have someone other than the test author complete a peer review of the updated tests and update the peer review section of the Peer Review Log . Remember any peer reviews that result in unfixed issues, should be added to the Issue Log.

Execute your integration tests and record the results in the Integration Test Log.  Note that you do not need to specify the "-b 200" flag for integration tests. 
For this project, you must pass (0 failed assertions) *all* integration tests, including those you wrote for this project.

4. Peer review

For this project, you should peer review all of your updated integration tests and your acceptance test generator. Additionally, if you have not been peer-reviewing your monitor designs, you should add peer reviews for them as well.

Team Design Portfolio

The portfolio you submit should contain the most up-to-date design package for your elevator system organized and formatted according to the portfolio guidelines.  You are going to update your portfolio every week, so be sure to keep an up to date working copy. 

Ensure your design portfolio is complete and consistent.

The following is a partial list of the characteristics your portfolio should exhibit:

Handing In Results

Grading Criteria:

The Minimum Requirements spreadsheet is located here
This project counts as one team grade. Points are assigned as follows.  A detailed grading rubric is available here (PDF).

This project assignment is worth 130 points:

Each team member must satisfy the minimum stated per-member requirements (e.g., one object for each activity). Team members who omit any required per-member activity will be penalized as described on the course admin page.

Back to course home page