Chapter 3: Methodology
The Audit undertook qualitative and quantitative research and conducted an extensive review of all documentation which provided evidence of the implementation of recommendations.
The Audit spoke to undergraduates, all members of the Review Implementation Team (RIT),1 members of the ADFA leadership team, military staff, academic staff, padres and international cadets. Initial interviews and focus groups with undergraduates and staff took place between 15 and 18 October 2012. Follow up interviews with key staff occurred throughout the Audit period.
Qualitative data
The Audit team held:
- 4 staff focus groups
- 12 undergraduate focus groups
- 50 individual interviews.
Focus groups with cadets included:
- first-years only across each Service
- second-years only across each Service
- third-years only across each Service
- Army cadets
- Navy midshipmen
- Air Force cadets
- International students
- Women-only groups.
Quantitative Data
The primary quantitative data used by the Audit was the 2012 ADFA Unacceptable Behaviour (UB) Survey.
Recommendation 25 of the ADFA Report states that:
ADFA develop and annually administer a survey in order to more accurately measure the level of sexual harassment and sexual abuse among cadets. This survey should be followed up with a strategic organisational response by the Commandant, with feedback provided to cadets and staff to ensure that they have an investment in any reform arising from the survey results.
The Unacceptable Behaviour Survey was extensively revised in 2012. The revised Unacceptable Behaviour Survey was administered to ADFA undergraduates and RMC Duntroon cadets by the Directorate of Strategic People Research (DSPR) commencing 20 September 2012. DSPR’s ADFA report was made available to the Audit on 21 December 2012. The Duntroon report was made available to the Audit on 14 March 2013.The Audit undertook its own analysis of this data.
Written submissions
The Audit invited written submissions between 1 October 2012 and 15 December 2012. The call for submissions was advertised through ADFA and the ADF and was placed on the Australian Human Rights Commission website.
During consultations with undergraduates and staff the Audit team provided cards with the website and phone details for people who wished to make contact with the Audit.
A toll-free hotline was also established to answer inquiries and to allow people to provide a verbal submission where they were unable to, or did not wish to, provide information in writing.
ADF documents
The Audit undertook an extensive request for information (RFI) process. 105 formal RFIs were made. All requests were coordinated through the RIT. This material provided evidence of implementation of recommendations, and useful information regarding relevant policies and practices, including complaints handling, incidents of unacceptable behaviour and attitudes of cadets and officers.
Observation
In 2013 the Audit team had the opportunity to:
- Attend the Staff Induction and Instructor Preparation Course program for 2013 – particularly those sessions that relate to Recommendations 14 a) and 14 b)
- Observe sessions of the Year One Familiarisation Training Program for the 2013 intake of undergraduates
- Observe the Sexual Offences Support Persons Course, 21 March 2013
- Observe the roll out of elements of the ADFA citizenship package in 2013 e.g. social media and e-safety, equity and diversity and alcohol and drug education
- Attend the Ethical Decision Making Seminar, 2-4 April 2013
Limitations to Audit
Qualitative data
As part of its methodology the Audit requested to conduct interviews and focus groups with ADFA staff and undergraduates in October 2012. This was agreed and was coordinated through the RIT.
The Audit was conscious that this was a busy time of the year but was keen to speak with undergraduates and staff before they left ADFA for the summer break and to understand their experience of the first 12 months of reform. This had been requested and agreed well in advance and all attempts had been made to minimise disruption.
There were some communication issues which impacted on the Audit team’s access to staff and undergraduates. These issues were resolved quickly and to the satisfaction of the Audit team.
Quantitative data
As noted above, a key piece of the quantitative data examined by the Audit was the Unacceptable Behaviour Survey. This instrument was significantly revised in 2012 and so the results of the administration of the survey in 2012 are not comparable to the results gained by the Review in 2011. The Audit commends the revision of the survey instrument and agrees that it is now a much more coherent and effective instrument. However, because of the changes to the survey, the extent to which the Audit can assess the changes at ADFA over the past 12 months in this area is very limited. More detail and an overview of the survey results is provided at Recommendation 25.
- The Review Implementation Team manages the implementation of the recommendations of the ADFA Report in collaboration with ADFA. It reports to the Commander, the Australian Defence College.