This is an HTML version of an attachment to the Freedom of Information request 'Defence.gov.au robots.txt'.


 
 
 
 
 
DEFENCE FOI 062/24/25 
STATEMENT OF REASONS UNDER THE FREEDOM OF INFORMATION ACT 1982 
1. 
I refer to the request by CR (the applicant), dated and received on 18 July 2024 by the 
Department of Defence (Defence), for access to the following documents under the 
Freedom of Information Act 1982 (Cth) (FOI Act): 
Documents that informed the decision to block Google's web crawler from indexing pdf, 
doc, docx, xls, and xlsx files contained on the site.
 
FOI decision maker 
2. 
I am the authorised officer pursuant to section 23 of the FOI Act to make a decision on 
this FOI request. 
Documents identified 
3. 
I identified no documents as falling within the scope of the request.   
Decision 
4. 
I have decided to refuse the request under section 24A of the FOI Act. 
Material taken into account 
5. 
In making my decision, I have had regard to: 
a.  the terms of the request; 
b.  relevant provisions of the FOI Act;  
c.  the Guidelines published by the Office of the Australian Information 
Commissioner under section 93A of the FOI Act (the Guidelines); and 
d.  advice  provided  by  the  Defence  Enterprise  Web  Directorate,  Enterprise 
Transformation and Governance Division, Associate Secretary Group. 
REASONS FOR DECISION 
Section 24A – Requests may be refused if documents cannot be found, do not exist or have 
not been received 

6. 
Section 24A(1) of the FOI Act states: 
(1) An agency or Minister may refuse a request for access to a document if: 
(a)  all reasonable steps have been taken to find the document; and  
(b)  the agency or Minister is satisfied that the document: 

(i)  is in the agency’s or Minister’s possession but cannot be found; or 
(ii)  does not exist.  

 
 



 
7. 
Paragraph 3.94 of the Guidelines advises the detail this statement of reasons should 
include to refuse a request under section 24A(l): 
…the statement  of  reasons  given to  the applicant  should sufficiently identify the 
document, explain why it cannot be found or is known not to exist or to be in the 
agency’s possession, describe the steps the agency took to search for the document, 
and note the limitations of any search… 

8. 
An extensive search for documents was conducted Within the Defence records 
management system (Objective) and other relevant personal/team files for records 
containing any reference to the phrase ‘robot.txt’. In addition, relevant personal and 
group email inboxes were searched for files containing the same phrase; however, no 
records matching the applicant’s scope were located. 
9. 
I am satisfied that all reasonable steps have been taken to locate the document/s sought 
by the applicant. I am satisfied that the documents do not exist, and refuse the request 
under section 24A(1) of the FOI Act. 
FURTHER INFORMATION 
10. 
As part of the normal course of business to deliver search engine optimisation for 
audiences, the robot.txt file was established to exclude .pdf, .doc, .docx, .xls, and .xlsx 
documents from Google search indexing. When not excluded, documents overwhelm 
the search results making page content far more difficult to locate. 
 
 
 
 
Sally Thauvette 
Accredited Decision Maker 
Associate Secretary Group 
Department of Defence 
 
26 August 2024