A new chapter for CDS: the CERN Repository

On January 7, 2025

The last weeks of 2024 marked a big step for the CDS team and all CDS users: the first migration of CDS content to a new version of the platform—we have just completed the migration of the first collection—Summer Student Programme reports. We can now consider the CERN repository a production service.

This moment is historic not only for our service, but also for CERN.

A bit of history…

CDS was established over 20 years ago as the institutional repository of CERN, tasked with archiving, preserving, and disseminating the organization’s research output, administrative documents, and multimedia. It is powered by the Invenio framework version 1, which has also been adopted by numerous repositories worldwide.

While CDS has dutifully served the CERN community for many years, it has become evident that it requires a refresh in terms of user experience and available features to meet the demands of 2025 and future.

Drawing from the successful experience gained with Zenodo, we have developed InvenioRDM: a versatile digital repository designed for researchers. InvenioRDM has been created collaboratively with over 20 partners around the globe, and incorporates all the best FAIR (Findable, Accessible, Interoperable, and Reusable) practices, offering a modern user experience.

The CERN repository: a new CDS

The CDS platform, currently accessible at https://repository.cern, serves as a new version of previous CDS website and is built on InvenioRDM.

Over the forthcoming months, our focus will be on tailoring it to suit the requirements of an Institutional Repository while integrating features specific to CERN. In parallel, we will migrate the content of each collection, one by one, until the end.

How?

We have developed a migration plan spanning 2024 and 2025, with a subsequent phase planned for 2026. This long and complex migration, set to unfold over several years, will be guided by three core principles:

Engage: before initiating the migration of any collaboration using CDS, we will actively engage with its members. Our goal is to ensure a smooth transition that meets the needs of each community without disrupting their work. We will thoroughly analyze use cases and collaboratively establish timelines.
Simplify: we aim to make submitting content easier and more intuitive. By empowering users with tools to independently organize and curate their materials, we will enhance the overall user experience.
Standardize: we will adopt standardized content metadata practices to align with FAIR principles and Open Science best practices

Next steps

When will my content be migrated? Where should I upload new documents? Who should I contact?
Don’t worry—these are questions we plan to answer together with you. We are committed to working closely with all content owners in CDS, gradually engaging with each group to share our plans and shape the future collaboratively.

Following the successful migration of the Summer Student reports, we’ve validated our migration processes and pipelines. Building on this success, we are now ready to tackle more complex challenges, with the next milestone being the migration of CERN Theses.

In parallel, we aim to explore the feasibility of bulk migrating content from small and medium experiments. Additionally, we plan to prototype a new review and comment workflow to address the needs of most users.

Keeping You in the Loop

This migration is an ongoing learning process, and we haven’t figured everything out yet! Your support and feedback are crucial to our success.

We’ve recently started documenting our migration journey and compiling information on a dedicated website, which is continuously evolving. Additionally, we’ll keep sharing updates and news right here.

If you have questions or concerns, don’t hesitate to reach out—we’re here to help.

Improved DOI, ORCiD, ROR and CERN integrations

By Karolina Przerwa

On December 20, 2024

In CDS-RDM

Our new CDS repository is now more FAIR and more integrated with CERN services. Read below how.

DOI – do I need one?

A DOI (Digital Object Identifier) is like a permanent address for something on the internet, such as a research paper, article, or video. It’s a unique code that never changes, even if the location of the content moves to a new website.

Think of it like a tracking number for a package: no matter where the package goes, you can always find it using the number. Similarly, a DOI helps you find a document online, even if its web link breaks or changes.

For example, a DOI might look like this:
https://doi.org/10.17181/dd19c-hwf65

When you click on it, it will always take you to the right place to access the content. This makes DOIs very useful for researchers, students, and anyone trying to find, share or cite information without worrying about broken links.

You do need, and you should prefer a DOI instead of other IDs, if:

the page of your document is public (the file might be restricted)
you want to keep permanent access to your article/paper/resource
you want any other researcher to be able to make references to your content (citations)
you want to share the content with the research community without worries about broken links

You do not need a DOI if:

your document is internal, fully restricted to to your collaboration
you do not need anyone to be able to cite your work/document
your document does not have research related nature

Choose your DOI option

Latest CDS upgrade, we introduced optional DOIs. When you are uploading a document, you can choose to:

Input an external DOI (for example, already provided by Zenodo.org, ArXiV.org or by a journal).
Obtain a new DOI, registered by CDS.
Avoid the creation of any DOI.

When you choose to obtain a DOI from CDS, all the information associated with your record, except the file content, will be registered with DataCite.org. This includes metadata such as the title, authors, and other details, which may be accessed and harvested by third-party services.

Find my colleagues

Since we are on the subject of submitting the content to https://repository.cern (aka CDS), how about providing the most accurate metadata possible, without hustle?

You can now easily find all of your CERN colleagues and external researchers with ORCiD profile when listing them as authors or contributors to your work.

But wait, what is ORCiD?

An ORCiD (Open Researcher and Contributor ID) is like a unique ID card for researchers (similar to DOI for research resources). It’s a special number that helps identify scientists, academics, and other contributors to research, so their work can always be linked back to them, no matter where it’s published.

Think of it like a social security number, but for researchers—it’s unique to each person and stays with them throughout their career. This is especially useful because many researchers might have similar names, or their names might appear differently in different publications.

For example:
An ORCID might look like this: 0000-0001-2345-6789

With this number, all of a researcher’s work—papers, data, and other contributions—can be easily found and correctly attributed to them. It helps avoid confusion and makes it simple to track their achievements over time.

Creators and contributors

We have automated the import of the ORCID database, which, at the time of writing, contains over 21 million ORCID records, and the import of CERN users with an account.

Where can you add your colleagues to attribute for their contributions? There are dedicated fields in the deposit form:

In the latest CDS platform upgrade, we improved the search capabilities and display of the autocompletion widget. You can now find your colleagues easier and add them even faster.

We highly recommend creating your own ORCID profile using this link. It is a must-have for any researcher!

Once you obtain and ORCID, you should connect it with your CERN Profile, using the Users Portal, as shown:

Finding affiliations

It is in a researcher’s best interest to provide the all possible metadata describing their research – more information they provide, higher the chance for other researchers to find about it.

Information about the researchers’ affiliations might be particularly important for highlighting the relationship with the organization which funded the research. Therefore, providing the up-to-date information about your affiliation will work well.

In CDS, we usually prefill that field for you, but our sources might not have the latest data, so you are able to fill the affiliations manually if needed. How? By using ROR identifiers (yes, yet another identifier!) or simply searching for the organization’s name.

A ROR identifier (Research Organization Registry identifier) is a unique, permanent ID used to identify research organizations worldwide. It ensures that institutions like universities, labs, and funding bodies are consistently and accurately recognized in research metadata, publications, and data repositories.

For example, many institutions have similar or changing names, which can create confusion. A ROR identifier solves this by providing a standardized, unchanging ID, like https://ror.org/01ggx4157 for CERN, which always points to the same organization.

ROR is open, free, and interoperable with other systems like DOIs (for publications) and ORCID (for researchers), helping improve the discoverability and tracking of research outputs. It’s a key tool for making research more connected and organized globally.

Rich metadata

We made sure to put in place all these integrations with ORCiD, DOI, ROR and the CERN databases in order to make it easy for you to fill out the most metadata automatically (and quickly!). We strive for seamless user experience, and we really hope that all these features will make your upload easier and… more complete!

Find most viewed and downloaded

Curious to know what are the most viewed or most downloaded records? You can now easily find it out by sorting searching results.

🎄☃️❄️ Best wishes and a happy new year!

The CDS team wishes you a joyful holiday season and a bright, successful start to the New Year!

Updates on the new CDS

By Javier Romero Castro

On October 10, 2024

In CDS, CDS-RDM

We’re excited to announce a set of new features and improvements to the new CERN Document Server (CDS), now running on InvenioRDM v12! Our focus has been on making CDS more user-friendly and efficient for everyone.

The long list of all changes is detailed in the full changelog. Below, the main highlights more relevant to the CERN community.

What’s new?

Enhanced Search Accuracy

We’ve improved the search functionality to make finding records easier and more precise. Whether you’re looking for specific datasets or publications, the search engine is now optimized to deliver more accurate results, helping you locate the content you need faster.

Mathematical Formula Rendering

For researchers working with complex formulas, CDS now nicely renders LaTeX formulas within search results and records.

Content Policy and Terms of Use

To ensure transparency and safeguard users, we have added a Content Policy and Terms of Use to the platform. These documents clarify the rules regarding content submission and usage on CDS, helping maintain a safe and collaborative environment.

Introduction of sub-communities

One of the biggest updates is the feature of sub-communities. Communities on CDS can now be nested, meaning a community can have a parent community. This change brings more flexibility in organizing and structuring data, catering to the diverse needs of departments and research groups.

User Interface Tweaks

We’ve implemented a series of UI improvements, including fixing issues with community logos and adding enhanced loading icons during login and logout among other fixes. These small but impactful updates make the interface more user-friendly and visually coherent.

Migration of the 1st collection

Under the hood, we are working hard to migrate the very first collection of documents from the current CDS repository to our new platform. We are now in the process of testing the migration of documents and files, and ensuring the correct redirection of web links.

This is a very important milestone: it will prove that the migration processes that we have put in place are working as expected, and it will unblock the migration of the next collection of documents.

What’s next?

Our team is already working on new features for future releases:

Collections

We are developing a way to categorize records easily within a community (or independently) based on metadata. This allows users to organize and navigate records more intuitively.

Automatic Ingestion of ORCID and ROR Values

To save time and streamline workflows, we are working on automating the ingestion of ORCID and ROR data into the system, ensuring that author and organization identifiers are up-to-date without manual input.

Integration with CERN users database

We are working on making CERN users findable when searching for authors or collaborators during an upload.

Stay tuned for more updates, and as always, feel free to share your feedback with us!

User experience updates on CDS Videos

By Zacharias Zacharodimos

On August 12, 2024

In CDS Videos

We are happy to share some new updates about the CDS Videos platform. Our latest infrastructure upgrade introduces several improvements to enhance performance, security, and user experience. Here’s a summary of the key changes:

Upgraded search software

We’ve transitioned to OpenSearch, a central IT service that significantly boosts our search functionality and ensures a better integration with IT-provided services, removing maintenance’s redundancy.

Platform and security upgrades

We’ve upgraded our system from CERN CentOS 7 (CC7) to AlmaLinux 9 operating system. This change was part of the CERN campaign to upgrade all systems running on the former software as it is not actively supported anymore. This change ensures our platform’s resiliency and provides a more secure environment for all our users.

Improved upload features

Parallel uploads: We’ve addressed and resolved previous issues with multiple large parallel uploads. Uploading several large files simultaneously is now faster and more reliable.

Upload form improvements

We have made some changes in the upload form based on feedback we gathered from you. We hope they improve how you upload content on our platform.

Keywords: You can now enter keywords separating them with commas. That will allow you to paste e.g. “keyword1, keyword2” and the system will automatically convert them into 2 separate tags for you after pressing ENTER, improving the process of tagging and categorizing videos.

Date: We’ve enhanced the date input functionality to allow you to input dates directly instead of using always the date picker widget. That is useful when you want to input a rather old date.
Contributors:
- When you enter a custom contributor name, the system will convert the name automatically to a capitalized version e.g. “doe, joe” will be converted to “Doe, Joe“, ensuring a consistent appearance across the platform.
- We’ve added a new contributor role, “Subtitles by” so you can give proper credit to those who provide subtitles for your videos.

These upgrades are part of our continuous efforts to improve the CDS Videos platform. We hope these changes will provide you a better, more efficient experience.

Updates on ILS and CERN Library Catalogue!

By Saksham Arora

On July 4, 2024

In CERN Library Catalogue

Over the past six months, we’ve been busy making the CERN Library Catalogue even better, and we can’t wait to share all the exciting updates with you.

All these improvements have been designed and implemented in close collaboration with our colleagues from the CERN Library.

News for the patrons

CERN Maps integration

Thanks to newly implemented CERN Maps integration, you can now precisely locate where a book’s shelf is within the library, making it easier than ever to find them:

A big thank you to the CERN Maps team for the successful collaboration, which made it possible.

Self-Checkout

We’re introducing self-checkout! Soon, you will be able to check out books all by yourself without requiring a librarian at the desk, by using the new in-browser barcode scanner from your mobile phone.

Other user interface and user experience improvements:

Books’ subtitles are now visible on the search results page.
The opening hours page now includes a link to Library location on CERN Maps.

Moving on to the other updates…

We have worked extensively on improving the search and content display:

Edition, publication year, and volume are now visible on book titles.

We’ve added a “More on this subject” section to highlight related books.

We have improved search capabilities to provide more accurate results:

Accurate results for searching without accents and special characters for books with accented letters or special characters.
Better search results with less ambiguity.

And guess what? You can now find and access audiobooks and online video courses in the catalogue!

For librarians

Bulk import for new content

Librarians are now able to bulk import audiobooks and online video courses from external providers.

The bulk importer navigation is now improved as well as document matching, which helps to reduce false duplicates and keep the catalogue clean when importing.

Library management

Interlibrary loans are now easily accessible in the back-office. Loan management is now easier, thanks to integration of closure dates (bank holidays, etc.) to all calendars displayed when choosing the start date of a loan. We’ve also improved on several navigation elements, making important metadata mandatory and added more metadata to better describe a book.

Looking Ahead

Our focus has been on making it easier to find and access books and other resources available in the CERN Library Catalogue, by improving how books are managed and displayed, enhancing checkout and search capabilities.

We hope that you will enjoy all these new features!

The new Web Lectures video player

By Javier Romero Castro

On June 26, 2024

In CDS

Today is an exciting day at the CDS website! We’ve been working hard behind the scenes to bring you some exciting improvements to your user experience. As of this moment, you can start enjoying the benefits of these enhancements:

Brand new Web Lectures player: thanks to the fantastic work of our colleagues from the Webcast and Recording service, we’ve said goodbye to the previous Web Lectures player (based on THEOplayer and JW Player) and welcomed a new video player, powered by Paella. This sleek, modern player is designed to make your video-watching experience smoother and more enjoyable than ever before.

Seamless Access: You’ll notice a significant change in how you access the restricted Web Lectures. No more redirections or weird pop-ups asking for credentials—restricted videos now display seamlessly, thanks to the integration with the CERN Single Sign-On (SSO).

Enhanced Download Options: We’ve always offered download options, and now it’s even easier to distinguish between lecture slides and presenter recordings. Also, subtitles are now available for download as well.

Smoothly Embedded Videos: If you embed videos from the CDS platform, you’ll experience a seamless transition to the new video player.

A Post-Mortem analysis on the recent DDoS attack

By Nicola Tarocco

On April 25, 2024

In CDS

In past days, the CDS website experienced an unprecedented Distributed Denial of Service (DDoS) attack. A DDoS is a malicious attempt to disrupt the normal traffic of a targeted server by overwhelming the target with a flood of Internet traffic. You can find here more information.

What happened

On Monday, April 22, 2024, at approximately 11:30 AM, our monitoring systems detected a significant increase in traffic to the website. Analysis of the incoming HTTP requests revealed a Distributed Denial of Service (DDoS) attack. As a result, the CDS website became inaccessible to most users.

As an initial response, we promptly reached out to the CERN Computer Security Team for assistance. Furthermore, we posted an announcement regarding the incident on the CERN Service Portal Status Board, referencing number OTG0149709. Additionally, we communicated the incident via our official Mattermost channel, accessible to CERN users, and we added an informational banner to the website for those who were able to access it.

First actions

We have quickly realized that mitigating the attack would take longer than anticipated. At 12:30 PM, we made the decision to restrict access to the website solely from within the CERN network. This measure ensured that CERN users could still access the website while allowing us to concentrate on implementing countermeasures.

Around 3:30 PM, it appeared that the attack rate had decreased. In collaboration with CERN Computer Security Team, we made the decision to reopen access to the website from outside the CERN network. However, less than an hour later, the attack resumed, with an even higher volume of traffic. We decided to close access again.

Resolution

Due to the overwhelming majority of incoming requests originating from a specific geographical location, we made the difficult decision to block access to the website from that entire area. Simultaneously, we reinstated access from outside the CERN network. This countermeasure was implemented on Tuesday, April 23, 2024, at approximately 3:30 PM. As an additional security measure, we completely disabled IPv6 connections. All operations have been performed by CERN Computer Security Team in collaboration with us and the Network team.

The restrictions on users accessing CDS from certain locations will remain in place until we confirm that the attack has ceased. We are continuously monitoring incoming traffic in order to lift these restrictions as soon as possible.

The reasons and specifics behind this attack targeting CDS remain unclear. We have provided all available logs and information to the CERN Computer Security Team, who will conduct the necessary investigations and take appropriate actions.

Next steps

As it is the first time that we experience such a large-scale and distributed attack, it’s evident that we were unprepared. However, this experience has provided valuable insights and lessons for both our team, the CERN Computer Security and Network teams. We’re actively leveraging these takeaways to enhance our infrastructure and ensure readiness for any future occurrences.

While the CERN Computer Security and Network teams are currently analyzing logs and enhancing detection and mitigation tools to accelerate response times, our immediate focus will be on improving our alarming systems. Additionally, we are prioritizing enhancements to our DDoS protection mechanisms. Furthermore, efforts are underway to establish a reliable internal infrastructure as a contingency in the event of external compromise, ensuring continued access to the website for CERN users.

More technical details

During the initial stages of the DDoS attack, we observed a traffic volume of roughly 5,000 requests per minute. However, the incoming traffic within the CERN network was constantly increasing (~~we observed the number of 20,000 requests per minute, and growing~~ UPDATE: we observed the number of 3.5M requests per hour).

While these figures may not seem excessively high, the CDS infrastructure is not designed to handle such volumes, as we aim to avoid over-sizing the infrastructure when unnecessary. By comparison, traffic on CDS typically reaches around 500 requests per minute, with peaks of 1,000 requests per minute.

Despite implementing counter-measures such as blocking numerous IPs or scaling up our infrastructure to accommodate more traffic, the number of requests continued to escalate during the attack. It appeared that the attacker was capable of increasing the size of the attack.

A sophisticated attack

Implementing counter-measures for attacks of this scale is challenging. The attack vector exhibited a high level of sophistication.

Here is an example of a single HTTP request (with the IP address masked):

<masked ip> - - [23/Apr/2024:12:59:59 +0200] "POST /6270607l7c07z7ldmt031x/6270607l7c07z7ldmt031x-6270607l7c07z7ldmt031x/ HTTP/1.1" 404 "-" "Mozilla/5.0 (Linux; U; Android 12; V2027 Build/SP1A.210812.003; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/123.0.6312.118 Mobile Safari/537.36 OPR/77.0.2254.69831" 0 1246 16756

As you can observe, identifying a consistent pattern to safely and effectively distinguish between legitimate and malicious traffic is challenging due to the following reasons:

The URL path and HTTP verb appeared to be entirely random, with most requests resulting in a 404 error.
The User-Agent was generated randomly.
We detected over 46,000 different IPs originating from various locations.
Each IP was responsible for a relatively low number of requests.

It was also unexpected to discover that attempting to block a large number of IPs could put pressure on many software components in the infrastructure.

Conclusion

Access to the website was restored within a few hours, successfully stopping the attack. However, this DDoS attack is still ongoing, and access to CDS from certain locations will remain blocked until it stops.

It is now even clearer to us that defending against such attacks requires a high level of expertise and investment in robust infrastructure and tools.

As service providers, we are grateful for the expertise and competence of the specialized teams at CERN. Their dedication ensures that we can effectively address challenges and maintain the reliability of our services.

The new Zenodo is live!

By Nicola Tarocco

On November 15, 2023

In CDS-RDM

In the entire 2023, until October, our team worked in closed collaboration with the Zenodo team to launch the new version, now based on InvenioRDM, the turn-key research data management repository platform.

You can read more about this very important milestone in the official blog post and the OpenAire blog.

The future CDS

This is also a fundamental step for the future version of CDS, which is also based on InvenioRDM. Thanks to this new Zenodo launch, InvenioRDM is now a battle-tested platform, and it will receive constant improvements to make sure that it fulfils the needs of researchers worldwide.

We have learned a ton preparing the new version of Zenodo, not only developing features, but also preparing the infrastructure. With all these lessons-learned, the new CDS will be a more reliable and performant platform.

Next steps

We will work until the end of this year 2023 to analyze the features available today in CDS, and identify the ones that are essential to migrate to the new version.

We are working on a detailed migration plan, and we will get in contact with the main communities to better understand their needs and ensure a smooth transition from the current CDS to the new one, in 2024.

We are very excited, and we are looking forward to seeing the new CDS being used at CERN!

Updates on the new CDS

By Nicola Tarocco

On June 28, 2023

In CDS, CDS-RDM

Summer has already started 😎 and, in the previous months, we have worked hard to integrate the latest development in the new CDS platform.

The result looks beautiful!

The new CDS platform is the brand-new version of the current CERN institutional repository, a modern and easy-to-use website where CERN users can archive and share their research, multimedia content or departmental documents.

You can now preview and try out the latest features in our test instance https://sandbox-cds-rdm.web.cern.ch (reachable from inside CERN campus). Just to mention a few, we have integrated users and groups CERN databases; newly uploaded publications will now have a DOI out-of-the box, ready to be shared and cited; files are securely stored in EOS file system. And there is much more.

The “Browse” section contains links to collections and categories to the former CDS platform: we will slowly migrate data to this brand-new CDS.

The footer of the new CDS website contains useful links to make sure that you will find the information that you need.

The production instance https://new-cds.cern.ch will be soon start to be used by some selected communities at CERN, and we will gather feedback to continuously improve it and make it as easy as possible to use.

After summer, more features will be coming 🚀: we will make it very easy to restrict and share documents with other users, and we will work on the administration panel to fully manage records and users in the system.

This version is just the base for the future CDS. More features will be needed to support all current use cases. To that end, we will be contacting and working together all main users so that we can define together the plan for completion of this future Institutional Repository.

If you wish to, open the new CDS website, login, try it out and share feedback with us!

The new CDS, based on InvenioRDM

By Nicola Tarocco

On January 12, 2023

In CDS, CDS-RDM

With the LTS release (v9) and the latest release (v10), InvenioRDM has reached the maturity needed for production-ready digital repository websites. InvenioRDM is a generic data management repository, developed by our team in collaboration with many partners all over the world. Free to use and open-source.

As already done by several partners (e.g. Caltech University, TU Graz University, TU Wien University), our team worked hard to create a preview version of the future CDS, available at https://sandbox-cds-rdm.web.cern.ch.

The new CDS website, based on InvenioRDM.

As first milestone, we have created and deployed the new instance of CDS and also migrated a selected set of records, metadata-only. This initial setup will allow us to iterate with the process of data migration, expanding incrementally the number of records and improving the data quality.

In the first quarter of this year, we will continue working on the InvenioRDM product, adding more features and integrating them in the new CDS website.

We will also start an analysis of the feature-set available in the current CDS, but still missing in the new platform: thanks to this, we will be able to come up with a plan for the next steps.

We are very excited to finally see the new CDS taking shape! Stay tuned for future announcements!