sample robots files

This commit is contained in:
jessop 2020-09-22 10:37:18 -04:00 committed by Sean Yesmunt
parent ba0c22e080
commit adaee548e2
3 changed files with 6 additions and 0 deletions

2
.gitignore vendored
View file

@ -23,4 +23,6 @@ package-lock.json
/web/.env.defaults
/custom/*
!/custom/homepage.example.js
!/custom/robots.disallowall
!/custom/robots.allowall
.env

2
custom/robots.allowall Normal file
View file

@ -0,0 +1,2 @@
User-agent: *
Allow: /

View file

@ -0,0 +1,2 @@
User-agent: *
Disallow: /