16. Assessment 2#
16.1. Software Engineering for Reusability#
16.2. Background#
The value iteration algorithm is a means of determining the optimal policy for a decision process that can be modelled as a Markov Decision Process (MDP). The main objective of this assessment is to implement the value iteration algorithm in python and host it it on github along with supporting material.
Your work should be sufficiently well designed and documented so that the assessors can use your documentation, examples, and package to successfully optimise a small problem modelled as an MDP. The details of this problem are NOT provided as part of the assessment.
16.3. Objectives#
The objectives of this assessment are :
Implement the value iteration algorithm in python
Package the implementation and host it in a github repository
Provide supporting material for your package in the repository
16.4. Suggested Approach#
Study sections 9.5 and 9.5.1 of Artificial Intelligence: Foundations and Computational Agents 2nd edition
Implement the value iteration algorithm using the pseudo code in figure 9.16 in section 9.5.2 of Artificial Intelligence: Foundations and Computational Agents 2nd edition.
Study and solve (at least partially) by hand the problem introduced in exercise 9.27 in section 9.5 of Artificial Intelligence: Foundations and Computational Agents 2nd edition.
Check that you can specify and solve exercise 9.27 using your implementation of value iteration.
Package your implementation and host it on github
Add supporting information, examples, and documentation to your repository so that another researcher is able to
reuse your implementation of the value iteration algorithm
rerun your examples and reproduce your results
implement code to represent their own MDP problem and optimize it using your implementation of value iteration
It is highly recommended that you look at similar projects hosted on github and/or PyPI for ideas of how to effectively realise item 6. For example
Try and reproduce some of the examples from these repositories with view to learning what works well and, importantly, what does not.
16.5. Assessment Criteria#
Your submission will be assessed according to the following criteria :
Quality of code : the extent to which your code is readable and uses appropriate data structures and design patterns
Documentation : the extent to which the package is well documented
Use of software engineering tools : the extent to which software engineering tools such as version control and unit testing have been appropriately used
Solution : the extent to which the code is reusable for other problems
16.6. Submitting your work#
Formal submission of the work should be made by inviting Daniel Grose (github name grosed) and Jamie Fairbrother (github name fairbrot) to be collaborators to your github repository by the submission date. .
Note, only files committed by the submission date will be considered for assessment unless other prior arrangements have been made.
Your github invitation should be sent by 17:00 on Friday 28th March.