Saving Digital Pathology

Every so often, I stumble upon some variation of the graph displaying American music industry revenues by format: vinyl is replaced by cassettes, which are replaced by CDs, which are (briefly) replaced by digital downloads and ringtones, which are finally replaced by streaming services [1]. Today, essentially all casual music listening happens through streaming (and mostly through giants like Spotify, Apple Music, or YouTube).

Why would it be any other way? Achieving the convenience of listening to almost anything, anywhere through other means than a large commercial streaming solution is possible [2] but out of reach for almost everyone. Streaming really seems to be unbeatable, especially for background music: revenues are increasing year-over-year even as official Spotify playlists at the top of the charts are filled with slop [3].

And yet, CDs are still generating half a billion dollars annually in sales in the United States, digital downloads (and especially piracy) are alive and well, and vinyl sales have been increasing every year since 2007. Who is "still" using these formats? Sure, there are nostalgics, audiophiles, cord-cutters, but also musicians themselves, producers, remixers, DJs, and collectors who are worried about their music disappearing from streaming services – essentially, anyone who wants to be a bit (or a lot) more than a passive listener.

---

Most clinical pathology labs in Canada are still living in the vinyl era. The vast majority of specimens sent to pathology laboratories for analysis go through a technical process where tissues are preserved in blocks of paraffin wax, which can then be cut onto glass slides; these slides are then directly examined under the microscope. At a minimum, digital pathology involves creating a (very large) image from each glass slide, which can then be viewed and manipulated on a computer display instead of a microscope. This is an expensive and complex proposition because slide digitization is tricky and labour-intensive, and there is no efficient way to handle the resulting need to store and display huge amounts of image data. Most laboratories in Canada, which are perennially cash-strapped and personnel-depleted, have unsurprisingly not made the jump.

But there will be no choice to move on eventually and embrace digital pathology. I don't intend this to be a piece about artificial intelligence. At this point in time, it isn't exactly clear how AI will transform (or replace) pathology practice. But it is certainly possible that not using AI will soon be an unacceptable practice when it can benefit patient care [4]. Most clinically relevant AI tools that have been published or commercialized certainly require a robust digital pathology pipeline. Besides, there is a lot of money to be made with digital pathology, especially for large cloud service providers (such as Amazon and Microsoft) that have the attention of those making decisions.

To be clear, I am not skeptical at all about the benefits of digital pathology. New avenues open up for teleconsultation, interdisciplinary treatment conferences, and quality assurance that would simply not be possible otherwise. And that is only for direct patient care activities. It is hard for me to imagine conducting clinical research in pathology without digital slides, or to maintain and share slide collections destined for pathologists-in-training by shuffling around boxes of fragile glass rectangles. My question is whether we will be able to choose to do all of these things if digital pathology goes the way of Spotify, or if those of us who want to share, remix or innovate will have to become like the modern-day CD collector or music pirate.

I think it is almost certain that most laboratories will end up with a software-as-a-service (SaaS) model for their day-to-day practice in digital pathology, similar to music streaming. Setting up an on-premises, or at the very least laboratory-controlled workflow for digital pathology is possible but difficult. It is something the leaders in the field have done (most notably the University of Michigan), at the cost of committed institutional leadership, in-house expertise, and a few million USD in spare change [5]. But for the rest of us, digital pathology will likely mean entering into SaaS contracts with tech giants. In the words of Ed Zitron [6]:

SaaS is one of the most dominant business models in tech, because it fits both the customer profile of "not wanting to run a bunch of infrastructure" and the tech industry's love of trapping people in distinct ecosystems that are hard to escape.

There is nothing inherently wrong with SaaS, but it's important to remember who will be making the decisions and what their priorities are. In Canada, funding for in-hospital activities is essentially entirely governmental, and decision-making for these kinds of transformative projects is heavily centralized. The priority for purchasing decisions is the delivery of clinical care to the population (understandably), and any particular secondary institutional mission such as research or education is generally an afterthought (or absent altogether).

Although I would love to be proven wrong, I don't see any particular reason why features like portability, data accessibility, or API access that are crucial for innovation, research and education would make it very high in a tender for digital pathology services. These are features that will have to be paid for by research funds and educational institutions, if they can even afford them at all. SaaS providers will be more than happy to upsell their customers to meet those needs later, which is a core characteristic of their business model [6].

All of this isn't just wild speculation, because it has happened before with another technological revolution in medicine: electronic health records (a.k.a. EHRs or EMRs). As excellently recounted elsewhere [7] by Robert Kuttner, some of the most innovative and well-known hospitals in the United States, such as Beth Israel and Brigham and Women's, had designed their own in-house EHRs, primarily oriented around the needs of end-user staff and patients. In the meantime, the majority of hospitals in the country were slowly swept up by Epic, the EHR system that now operates a near-monopoly.

Epic arguably attained its dominant market position not by making better or more interoperable software, but rather by enabling hospitals to maximize billing for clinical encounters (at the expense of efficient clinical encounter documentation). Even Beth Israel and Brigham eventually had to make the switch. Of course, all of the data Epic has access to thanks to its paying customers isn't generously given back, but sits behind fences and barriers, including for researchers at their home institutions.

A future where glass slides are shipped off to a Digital Pathology as a Service provider, never to be seen again except through a bare-bones web application (with a few subscription AI tools), isn't so far away [8]. While this certainly wouldn't be great for clinical tasks, it would be absolutely terrible for research. Losing control over the raw materials of pathology would probably end any innovation coming from inside the lab (as the ceiling for what can be accomplished would always be dependent on whatever the provider is offering).

Although I am perhaps overdoing my musical metaphor, the experience of practising pathology would come to more closely resemble listening to Spotify's official smooth jazz playlist, aimlessly wandering between artists that do not really exist [9], rather than building and organizing a collection year after year, becoming more familiar with each record, and sharing it with others. We stand to lose a great deal if we become divorced from the materials of our work (and pathology is very much about the physical tissue we handle). Being able to explore fleeting ideas that could turn into research questions or experiment with tools others are sharing online is the start of opportunistic, low-cost translational research. Instead, it could simply never happen.

---

So what do I mean by "saving digital pathology"? I hope I have made clear already that we need to be deliberate about not losing out on much of its potential, but I also mean it in the more literal, floppy-disk icon sense. Even tech giants discontinue services with loyal followings [10]. Ostensibly public and permanent scientific data gets deleted or hidden [11, 12]. There is no guarantee that today's SaaS provider won't pull the plug tomorrow, or if your digital pathology data will be kindly returned to you should that happen (never mind that you may not have the infrastructure to store it). You can leave Spotify, but you can't take the music with you (or even export your playlists).

Most laboratories will not be able to set up an on-premises workflow to generate and save massive amounts of digital pathology data, let alone develop a user-friendly workflow in-house to manage specimens from reception to sign-out of the final report. Nevertheless, there are some low-hanging fruits that I think are possible even with limited resources. These are oriented along three important axes of modern pathology practice: generation and retention of visual specimen artifacts (gross photography and slides), the value-added activity of pathologists, namely diagnostic interpretation, and daily informatics tools that often exist outside the EHR or the laboratory information system (LIS).

Saving images

Images are (still) the bedrock of pathology. Non-visual information in pathology (such as molecular or genetic testing) often relies on visual interpretation to select the right samples for examination or to properly interpret the results. Image data, especially microscopic photography of samples stained with the cheap and ubiquitous workhorse stain hematoxylin and eosin (H&E), are remarkably rich for state-of-the-art artificial intelligence applications. Being able to extract, modify, and manipulate images is as important to innovation in pathology as manipulating the tissues themselves.

Although most labs will struggle with saving all of their image data locally because of the demands in terms of file storage, and the labour required to back it up and manage access, it is possible to prioritize data that is either irreplaceable or of high diagnostic, scientific or educational value.

This is the case for almost all gross photography (images of specimens as seen by the naked eye). Usually, whole specimens that have been removed from patients are not kept intact or kept at all. Fortunately, these types of images are usually taken with regular digital cameras and result in fairly small file sizes, which can be handled by consumer-oriented solutions. For gross photography, a large lab that handles tens of thousands of complex cases every year would probably require as much storage infrastructure as a family of dedicated amateur photographers.

As I have already mentioned several times, whole-slide images (WSI), which are digital "scans" of glass slides that are made by stitching microscopic photographs together, are more difficult to handle because the file sizes are huge by comparison to regular photographs. Some WSIs can exceed 5 gigabytes in file size. As of today, the infrastructure and cost of keeping digital copies for hundreds of thousands of these files every year is usually prohibitive, except for labs with considerable resources and technical expertise. Luckily, a large proportion of these WSIs is at least partially replaceable (glass slides can be scanned again to generate WSIs, and completely new slides can be made from archival material left behind). It is also usually possible to know in advance which WSIs will be more "valuable" for future use and which can be deleted if necessary.

Some (most?) digital pathology services already offer the option to mark these more valuable WSIs in advance, so that they are easier to retrieve later. But if the copies only ever live within a cloud or walled service and have to be retrieved through it, labs still run the risk of data loss, service shutdowns, barriers to access at scale and unanticipated access monetization.

I hope that most labs that convert to a full digital pathology workflow maintain at least some method of keeping copies of selected WSIs as regular files accessible using a general-purpose computer. For smaller labs, this could be at most a teaching collection of a few hundred to thousand cases. Larger labs or those with a research mission should be a bit more ambitious and try to preserve WSIs from select types of cancer cases (usually first-time diagnoses and surgical resections for a few organ systems), since they are a low-hanging fruit for translational research efforts.

Simple solutions like consumer network-attached storage (NAS) are probably sufficient for most use cases and are relatively inexpensive. Sticking to default configurations minimizes data loss and security risks. Additional backups are nice-to-haves but not absolutely essential, because the files are usually ultimately replaceable or retrievable from the clinical cloud service. Fancy redundant configurations like RAID are a money sink because uptime is usually not a very important consideration in this scenario (and RAID is not a backup). Use your money to buy more storage!

Besides file storage, there are a few other challenges to look out for. WSIs are confidential patient data and need to be treated with the appropriate precautions. It is probably best to handle WSI retention for clinical purposes through the vendor's digital pathology product (such as slide review or future interdisciplinary conferences where treatment decisions are made). This use case will probably have been planned and budgeted for at the outset.

If WSIs are being kept for research or teaching purposes, it is not necessary to preserve patient-identifying metadata (for example, the name, date of birth, medical records number, date of scan, hospital). It is best to remove these as soon as possible from the WSI. I like to use wsi-anon, an open-source (MIT-licensed) library that has been published for this exact purpose [13].

Once all identifying data has been removed, it's probably sufficient to treat these WSIs like any other sensitive file within the lab or hospital IT system. Healthcare institutions are notoriously bad at security, but I don't have much more faith in cloud storage providers either (ironically, "security" is often a major selling argument for SaaS solutions, which are themselves prime targets for Ransomware-as-a-Service providers [14], more so than individual institutions).

If an attacker does manage to get a hold of WSIs, there are theoretical identification attacks possible [15], but these are honestly inconsequential compared to what can be achieved with unauthorized access to hospital systems. Removing sensitive metadata and keeping files inside networks that someone should already be responsible for securing is probably good enough.

Saving diagnoses

Pathology is useful to patients and their treating physicians because its practitioners synthesize a variety of medically useful findings into a written report. For the same reason, pathology reports are an invaluable source of data for research, even absent the corresponding images. In addition to diagnoses themselves, reports often contain information about the underlying biology of diseases, or specific variables that distinguish cases of the same disease from each other.

Essentially all labs, whether or not they have fully transitioned to digital pathology, have an electronic LIS that stores these reports, and sometimes individual variables in a more structured fashion. However, these capacities vary. Synoptic reports, which are human-readable structured text that conforms to a key-value structure, are integrated differently: some systems store the key-value pairs in a machine-readable way, while others will only keep the output as unstructured text, which makes retrieval more complicated. Even when data is organized intelligently in the LIS backend, extracting it may not be user-friendly (or may require monetized API access!).

Obviously, the ease of identifying and retrieving these data influences research (and to a lesser extent teaching) potential. Want to find all of the cases of colon cancer in the past year that had invaded beyond a certain depth? What about finding all the cases of a rare diagnosis in the last three decades? Depending on how one's LIS is built and delivered, these can be very easy or very hard queries to resolve. In my case, they are unfortunately hard!

So what does this have to do with digital pathology? It would be rather unusual to make such a fundamental change to the laboratory workflow without making any changes to the LIS. Digital pathology creates a whole new class of assets to track, retrieve, and bill for (even in universally funded systems where workload is calculated). What this means is that a big transition is an opportunity to make changes, hopefully for the better. There is even published experience in the scientific literature where a transition to digital pathology and an overhaul of synoptic reporting went hand in hand [16].

While it's obviously a good idea to seize any opportunity to improve things in the molasses-slow milieu that is healthcare, similar challenges as those I mentioned earlier for saving images will occur for reports. Any process to select a digital pathology-compatible LIS will inevitably focus on clinical needs first and foremost. With the possible exception of quality control tools, it is entirely conceivable that big data or research tasks may not see any improvement. Worse, SaaS-ification and cloud-ification could result in reports and diagnostic data that are even more walled-off than before.

In these cases, it may be necessary to resort to good old copy and paste. Reports can be generated outside of the LIS, saved or backed up in some fashion, then pasted into the report area for final verification. This even works for synoptic reports: there are many (simple) tools in active development [17] that allow the generation of these types of structured reports in a local browser. These need work to be able to interface with a local and secure database in a future-proof and lightweight way, but I think the effort is well worth the benefit for future data extraction or review. Even the lowest of low-tech systems (pasting final diagnoses into a spreadsheet) can end up being surprisingly powerful later. Just make sure to anonymize everything!

Saving future options

Relinquishing control of and local access to one's own data is one thing, but it is something anyone with an internet connection has been used to for a while. The cloud is, after all, just someone else's computer [18, 19] (or 10 000 of them [20]). This isn't a problem as long as a service performs well and is reasonably priced. Unfortunately, keeping customers by providing a great service at a fair price isn't as profitable as locking them in, then upselling them while making it as painful as possible to leave.

To be fair, this phenomenon isn't exclusive to SaaS companies, but these tactics are even more essential in a subscription-based fee model where expensive licenses aren't typically purchased upfront. The double whammy for digital pathology is that typically, very little action is required by vendors to make this happen, as digitized pathology labs rapidly produce huge amounts of high-importance data in such a way that exit or data transfer costs spiral beyond any given year's budget.

I hope that I have already convinced you why thinking ahead on saving (at least some) pathology materials is important. The harder step is thinking of the bigger picture. Beyond file storage, an LIS, and their various integrations into other software, there are many more unrelated tools that can end up being bundled in. Get used to QuPath [21] for WSI manipulation and OMERO [22] as a WSI server. Edit static images with GIMP [23] and analyze them with Fiji [24]. Use plain text and open-source databases whenever practical (and try not to store them in your digital pathology provider's cloud!).

When the day comes to change providers and your lab will have to negotiate an exit deal, you will be happy that your most important materials and unrelated tools or applications won't become bargaining chips you need to buy back.

---

I don't know what pathology will be like in 5 or 10 years, but if it still exists in a form that would be recognizable to me today, I hope that there will still be equivalents of vinyl collectors, remixers, and nostalgics: a few people still hanging on to their glass slide collections, with intimate knowledge of optical microscopy, and others busy modding and hacking around digital pathology tools. Even a small number of dedicated pathologists will be enough to keep another way of doing things alive and well, even if the bulk of clinical work will be done on the Spotifys of digital pathology.

In the meantime, I'll keep buying hard drives whenever they're on sale. ❦

---

References