Provision of web and social media archiving services
UNHCR (UN High Commissioner for Refugees), Geneve
UNHCR may award Frame Agreement with initial duration of 3 years, potentially extendable for a maximum of 2 periods of up to 1 (one) year each. The successful bidders will be requested to maintain their quoted price model for the duration of agreement.
Deadline: 14/08/2023 CEST
Technical requirements
I. Capturing data and remote harvesting
A. The Supplier will provide for the capture of periodic snapshots of the UNHCR web estate. UNHCR currently performs 62 website crawls, and 33 social media account captures on a yearly recurring basis, spread over 3 key dates in the UNHCR calendar and consisting of core sites from UNHCR’s web estate.
UNHCR may also require captures in addition to the usual yearly crawls. These would be performed on an ad hoc basis, usually in response to emergencies and other world events of concern to UNHCR and as these events unfold. Extra crawls may also be required in order to create a snapshot of a website before it is taken offline. The supplier should be able include these extra crawls, at short notice if needed, and should provide a pricing list for ad hoc crawls.
Please note that UNHCR websites are very different in sizes and can be in a wide range of languages and scripts/alphabets. Suppliers will be able to capture and replay all websites and social media accounts, regardless of language or script (e.g. Latin, Cyrillic, Kanji, etc.). Languages include but are not limited to; English; French; Spanish; Arabic; Russian; Chinese.
B. As part of the RFP process, all bidding suppliers are required to provide test crawls of the following website sections:
• https://data.unhcr.org/en/situations/ukraine
• https://www.unhcr.org/ar/about-unhcr
• https://emergency.unhcr.org/protection
and the following social media accounts:
• https://twitter.com/Refugees
• https://www.facebook.com/UNHCR/
• https://www.youtube.com/UNHCR
After the tender submission deadline, the test crawls will be pre-planned with the suppliers so that only one crawl is done at any given time. The Records and Archives Section will act as focal point and will nominate one person who will co-ordinate the crawls with the suppliers. The crawls will be evaluated based on completeness of the crawl. Suppliers are required to set up a temporary portal and provide the results to UNHCR for review.
C. Crawls will be domain or path level, as appropriate, and they are expected to be complete.
D. The required method of capture for websites will be by remote harvesting. Social media accounts should be captured remotely as well, without requiring authentication from the account owner where possible.
E. The politeness setting for the crawler will be agreed prior to crawling.
F. Any archive crawler will identify itself using a UNHCR Records and Archives agent. E.g., using archives@unhcr.org
G. The Supplier will generate metadata for each snapshot and for each digital object derived from a URL. The minimum information required for snapshot and WARC file is:
• Snapshot name
• Seed URL
• Time and date of capture
• Size (no of objects and volume)
• Content type (i.e. format)
• Object (e.g., ARC or WARC file)
• URL retrieved
• It must be clear which objects relate to which snapshots
• Response code
• HTTP response header
H. UNHCR will determine the scheduling of the regular crawls. UNHCR will communicate crawl lists and requirements to the supplier in advance of the crawls. Extra crawls may be required in response to world events affecting UNHCR and the supplier should be able to accommodate these extra crawls as needed.
I. The Supplier will use their own crawler and access tools in combination with open-source tools (such as Heritrix, Conifer, etc.)
J. The Supplier will demonstrate its capacity to archive social media. The minimum required is the capacity of archiving UNHCR Twitter and YouTube accounts which are already included in the agency’s Web Archive. UNHCR also wishes to include social media other than Twitter in its archive, in particular Facebook, Instagram and LinkedIn, or other platforms in future.
Consequently, the Supplier will have to demonstrate its capacity to archive these social media, and/or its R&D activities in this field.
Company Qualifications
• A description of your company with evidence of your company’s capacity to perform the services required, including company profile, registration certificate, security certificates, and last audit reports
• Year founded
• If a multi-location company, please specify the location of the company’s headquarters, and the branches that will be involved in the project work with founding dates.
• Experience in the business or the number of similar and successfully completed projects
• Any relevant experience working with UNHCR, other UN agencies and NGOs should be included. Please specify the number of years working with public NPOs.
• Number of similar projects currently underway. Please elaborate on these projects and provide proof.
• Number of research and development projects in the web archiving field.
• Total number of clients. Please provide a list.
• You are requested to provide three references that we may contact from your current client List
Leave a Reply