Edeltraud Aspöck, Helga Di Giuseppe, Elizabeth Fentress, Guntram Geser, Anja Masur, Jessica Ogden, Teija Oikarinen, Despoina Tsiafakis
In the ARIADNE project the Excavation and Monuments Special Interest Group, lead by Edeltraud Aspöck and Elizabeth Fentress, represents the needs of archaeologists and heritage professionals, looking at issues that particularly affect them. In the course of a Skype meeting by the group on the 28th of January it was decided to carry out a survey on the management and storage of excavation data in European countries, asking participants about their current practice.
184 answers were received to a Survey Monkey questionnaire from people excavating in 37 countries, of which the best-represented were Bulgaria (55 responses), Italy (23) Greece (15) Romania (13), Spain (13) and the UK (10). The distribution probably reflects the Fasti countries as much as anything else, as the survey was promoted on our Facebook page and letters to the contributing archaeologists.
Of these, 66 used a database of some kind, for some of their data. The next question asked for a detailed breakdown of what data was recorded where: here there were fewer answers, due to a glitz in the original question. By the time this was fixed the following answers emerged: if we combine paper and individual digital files, these form the basis for half of the storage of context data, 57% of photographs , 51% of plans, 40% of pottery and 41% of other finds, with roughly 62 responses. This means that slightly more than half of the material is actually recorded on spreadsheets, relational databases and ArcGIS or similar. This latter was most used for plans, of course, with roughly a third of the respondents using a GIS system. However, only 54 people answered positively to the question as to whether their database was relational: given that only 72 answered the question at all, this suggests that only 29% of the whole sample was using anything beyond a spreadsheet. This appears to be confirmed by the answers to the next question, as to the source of the software, which was answered by only 53 people. Of these, 40 used commercial or proprietary software, such as Microsoft Access and Filemaker, while 17 used open source, bespoke, or setups that combined commercial and open source.
38 people named the program they use: of these, 13 used Access, 4 Filemaker, 9 ArcGIS, 2 ARK and one each Intrasis, Digital Dig Team, MySQL and Blackpen. For 21 of these, or 11% of the sample, the data was web-accessible, although this response is slightly modified by the responses to the next question, which suggested that 31 respondents, or 17% of the whole sample, made their work available to excavation personnel until the work was published, while 21 or 11%, intended the databases to be open access at some point.
The final question asked for comment on the systems used. 39 responded. Dissatisfaction was expressed by people who had to fill in both paper forms and computer databases, without having remote access to the data. Respondents without web-based systems also complained about not being able to share data. Simple spreadsheets, e.g. MS Excel, were also seen as inefficient, where even an Access setup was seen as poorly integrated. The combination of AutoCAD and ArcGIS or QGIS was more successful, while bespoke systems, such as that used in UCLA and reached through the university servers, provoked enthusiasm, showing ‘issues we did not think about before’ . One of the users of ARK was satisfied, but noted that “it was initially rather tricky, and it is still not easy to play with the structure.” Server access was very important to satisfaction: even Filemaker was better-liked on the University server.
The final question, about where data was stored, provoked a very spotty response. 11 projects stored their data on University servers, while 5 were on national or museum servers. 12 were stored on personal computers or hard drives, and 2 on Dropbox. The silence of the rest was resounding.
Although the data are not numerous, several countries provide enough to say something more:
Austria (Edeltraud Aspöck and Anja Masur)
Altogether twelve archaeologists excavating in Austria responded to the survey – but five did not provide any further information beyond that they work in Austria. So, altogether seven archaeologists filled out the survey, two indicating that they excavate in countries other than Austria. The majority uses databases and paper records to document excavations. Only one indicated that they used only databases, another one to use paper records exclusively. Spreadsheets seem to be an exception as they are used in only one case. Photographs are all made digitally, but contexts, plans, pottery and other finds are recorded using paper forms – and individual digital files (equal shares). But, the same information that is stored in paper and/or digital files is also saved in databases. All but one use relational databases. Generally, most use software that is not open access software. Three archaeologists stated to use AutoCAD, one that they also use ArcGIS, two use Microsoft Access and only one indicated using Open Access MySQL.
Regarding web access of databases, only one replied that they will make data accessible after print publication. Three out of five answered that they were mostly satisfied with their excavation documentation system. Some find it difficult to use AutoCAD, which is prescribed by the Austrian Federal Monuments Office. Someone mentioned that they would be interested in an open-access/networking database, but they find that there is no system ready to use yet.
Regarding the storage of the data, three out of five store data on personal and/or institutional computers or servers. All archaeologists are obliged to deliver excavation data tothe Federal Monument Office within three months after the end of the excavation for archiving. Guidelines for data formats can be seen online: http://www.bda.at/publikationen/1042/17658/Richtlinien-fuer-archaeologische-Massnahmen.
Of the 17 who responded to the question where context data were stored, 2 used databases alone, 8 paper alone, and 10 a combination of the two. 7 used relational databases. Of these, one respondent stored data in the archives of the National institute of Archaeology, while four responded that it was stored on a pc. QGIS was the only open source software used.
Recording was more or less evenly split between paper records and some form of database, with only 5 respondents using both. 14 of the 24 respondents used relational databases, generally commercial or proprietary (ARcheodata in one case) but in four cases open source (ARK in one case). 6 sets of records were kept on University servers.
Finland (Teija Oikarinen)
A version of the survey was sent to a number of archaeologists, and got 17 responses, of whom 13 excavated in Finland. Excavation data is mostly recorded in individual digital files (12), also in databases (7), as well as often paper records are used (8). Also digitizing in digital files (5) and digitizing in databases are used (4). To the question as to whether the database was relational 7 answered yes, 5 no, and 5 either didn’t know or didn’t use a database. 12 used commercial or proprietary software (mainly MS Access, with MapInfo and ArcGIS, although QGIS and a bespoke system, designed for a museum, were mentioned. 8 replied that the database was web-accessible. Individual comments indicated a need for shared, web-accessible databases.
As to availability, in Finland all archaeological reports and archived material are public and managed by the National Board of Antiquities. Digital files and databases are a more complicated question. I do not know any project that has published original data or databases online.
Greece (Despoina Tsiafakis)
The data from Greece regarding excavation data management originate from 12 respondents, 3 of whom excavate in other countries, too. The data are rather incomplete since not all respondents gave answers to all questions. From the data received we conclude that most excavators (5) record their field data in both databases and paper records, while 3 more responded that they are just using the one or the other medium. Only 4 of the respondents are using relational databases, while 6 did not answer at all. Moreover, more than half of the participants (7) provided an answer to the question regarding the source of their software, with 5 of them saying that it is commercial or proprietary, 1 of them saying that it is open source and the last one saying that it is a combination (commercial or proprietary, open source and bespoke). When the participants were asked about the name of the program used, many well-known software were mentioned, such as Access, Filemaker, ArcheoData etc. The question about whether the database being web-accessible got mostly negative answers (5 out of 6) and only 1 positive. Only 2 respondents gave information about the people for whom the database is made available, with both of them saying that it is available for excavation personnel from the start of the excavation procedure. Moreover, 1 of them said that they provide open access to the database after post-excavation work and the other after print publication. The satisfaction of the systems users differs according to their answers. 7 stated that they were satisfied, but 2 of them raised issues like flexibility and time consumption, too. 1 said that he was not at all satisfied and 1 just stated “complex”. Last, the majority of the excavation data seems to be stored in both personal computers and/or university servers, while less often external drives are used or open access platforms. The need for data available in shared locations apart from the excavation director’s computer was underlined in the responses, giving a glimpse of a major issue in modern archaeological science in Greece, that of intellectual property over the material.
Discussion and Comment (Fentress)
The question of how representative the sample is is of course key, here: very few people responded from Britain, and it would be good to know more about practice there. But in general, we are looking at a sample of people in contact with ARIADNE participants, and thus theoretically more wired in. If this is the case the results are pretty dismal, and are what we might have expected in the 1990’s, but are a surprise nearly two decades later. It suggests that the technologically competent people are talking among themselves if not preaching to the choir, and that general practice on data management lies very, very far behind. Some of this certainly reflects economic issues, such as the cost of AutoCAD and ArcGIS, but much of it may simply be a question of what excavations directors are used to and feel is necessary: there is a lot of good, open source software out there, and servers cost less than computers – while Ethernet cables cost nothing at all.
My own experience is that the relative technological sophistication of my projects has come from the bottom up, introduced by the youngest members of the team. In a survey archaeology project on Jerba, where the need for a GIS was obvious in 1995, we sought a GIS-competent person outside Classical archaeology departments, and came up with an undergraduate from Buffalo. On his advice, the dig house was networked, and data input regularly onto Filemaker. By the end of the project there was a full-fledged GIS of the survey data. Supervisors on the next project, an excavation in Volubilis (2000-2005), suggested that there too we use an integrated, networked database, based on Access, which worked well. However the failure of Microsoft to make its next upgrade of Access capable of reading previous databases rendered that material almost impossible to use except on a carefully preserved computer, by now over 10 years old, and I would never again use their software for data management. The next excavation, Villa Magna (2006-2010) used an early version of ARK, adjusted for us by one of the developers. This was a pain initially, but manna by the time we finished, allowing the multiple participants in the project to prepare the publication from 4 countries in record time. Cemetery data from 400+ burials, in particularly, could not have been managed without it. In each of these cases the initiatives came from younger members of the project, and I am still grateful to Michael Frachetti, Guy Hunt, Luca Passalacqua and Andrew Dufton for their success.
The problem raised by Despoina Tsiafakis, above, about intellectual property, is of course a cultural and not a technical one. Much of it derives from the traditional hierarchical structure of excavations, and the relative position of technical assistants – or younger archaeologists in general, in that hierarchy. Innovation is far more likely to come from younger archaeologists and technical assistants. It is to be hoped that peer pressure, and, perhaps, more publicity as to the availability of open source software, will convince older directors to move towards making data from the excavation and from all specialists more easily accessible. Excavation should, of course, be a truly collaborative process, and a fair distribution of recognition is necessary.
Further, excavation data should be saved, archived and made accessible. The overwhelming conclusion from the survey was that, with the exception of Austria and Finland, where all excavation data has by law to be deposited with the Federal Monuments Office, storage of excavation data was a very private and indeed haphazard procedure. This is clearly not good practice: the data should be backed up and accessible through an online database, first to the excavation personnel, and then to the public. Various guides to good practice are available, e.g.:
– Archaeology Data Service / Digital Antiquity: Guides to Good practice. http://guides.archaeologydataservice.ac.uk/
– ARCHES: The Standard and Guide to Best Practice in Archaeological Archiving in Europe. Prepared by K. Perrin et al. EAC Guidelines vol.1, Namur 2014 (available in Czech, Dutch, English, French, German, Icelandic and Swedish), http://archaeologydataservice.ac.uk/arches/Wiki.jsp?page=Main
– Open Context: Guidelines for Web-based Publication in Archaeology. Prepared by E. Kansa and S. Whitcher E. Kansa, January 2011, http://ux.opencontext.org/blog/wp-content/uploads/2011/05/Guidelines_Jan2011.rtf
One of the problems for many respondents was clearly a lack of knowledge of available systems.. There are various systems available: commercially, and a very few available for free download. These are clearly not publicized enough, and the decision of the SIG, meeting during CAA in Siena, was to take the project a step further, and review available systems using a carefully structured pro-forma which would evaluate performance, availability and cost. The project will be led by Holly Wright, who will be communicating with us shortly as to how to go about gathering information.