How to Read Affine of a Dicom File

Sometimes you recall you understand something, but you fail to explain it. This is the time that you accept to await back from a dissimilar perspective and start over. When you dig in medical images y'all will see different concepts to seem vague and non-intuitive, at least in the commencement. Y'all will see people discussing DICOM and coordinate systems you accept never heard before. Every bit a event, a lot of misconceptions and confusions are born. If you are in this position, or if you would like to know about AI in medical imaging this article is for you.

Back in 2017, when I applied for my main's degree in biomedical engineering everybody asked me why, as I was already obsessed with deep learning. At present, every multidisciplinary deep learning enquiry projection requires domain noesis such as medical imaging. Interestingly, the funding in the AI Healthcare domain is continuously increasing. Every bit an quantitative example of first google search that one can observe out:

The market place for car learning in diagnostic imaging will top two billion $ by 2023.

So, the reason that I decided to write this article is to assist ML people dive into medical imaging.

In a previous article, I talked about a common deep learning pipeline applied to multi-modal magnetic resonance datasets. All of that of grade with our under evolution open source pytorch library called medicalzoo-pytorch. Even so, I didn't dive into the particularities of the medical world too much. In the end, I used already candy information from an ML competition (and non from a messy hospital), so somebody else did the muddy piece of work for me. This tutorial is partly based in the nipy [1] and 3D Slicer.org [2] documentations for medical images and Dicom files.

To dive deeper into how AI is used in Medicine, y'all can't go incorrect with the AI for Medicine online course, offered by Coursera. If you want to focus on medical paradigm analysis with deep learning, I highly recommend starting from the Pytorch-based Udemy Class.

However, I decided to adapt and revisit the concepts and brand them more than familiar to Machine and deep learning engineers. There are a lot of assumptions that ML engineers have no idea about. Other multi-disciplinary projects have this kind of terminology problem. To this end, I considered it of great value to span this gap betwixt medical imaging concepts and deep learning that no one talks near, in this humble post. At to the lowest degree, I'll try my best, ane concept at a fourth dimension!

For a more comprehensive approach on AI in medicine, bank check out the awesome brand-new coursera grade on AI for medicine.

Notation: medical image tutorials oft call the MRI and CT exams as 'model'. For convenience, and to avoid misconceptions we volition use the world modality, throughout this tutorial to refer to any kind of medical image exam. The word 'view' and 'plane' are used interchangeably.

We are all ready to go into the medical imaging globe!

The coordinate systems in medical imaging

A coordinate system is a method for identifying the location of a point. In the medical earth, there are three coordinate systems commonly used in imaging applications: the world, anatomical, and the medical image coordinate arrangement.

Medical-image-coordinate-systems Source: 3D slicer documentation [2]

Earth coordinate system

The world coordinate system is a Cartesian coordinate arrangement in which a medical epitome modality (e.k. an MRI scanner or CT) is positioned. Every medical modality has its own coordinate organization, but there is only one earth coordinate organization to define the position and orientation of each modality.

Anatomical coordinate system

The most important model coordinate system for medical imaging techniques is the anatomical infinite (also called the patient coordinate system). This space consists of 3 planes to describe the standard anatomical position of a human:

the axial plane
the sagittal plane
the coronal plane

It is worth mentioning that this system was invented for better advice between doctors and radiologists.

Tip: In order to sympathise the anatomical planes, you take to visualize a standing human looking at yous.

Moreover, in this arrangement everything is relevant. Thus, the 3D position is defined forth the anatomical axes of anterior-posterior (front end-dorsum) and left-correct and junior-superior, as we will see. By this sense, all axes take their notation in a positive direction.

Axial plane

The centric airplane is actually when yous place your point of view above the patient and look downwards. Depending on the region of the 3D medical image you lot volition observe different anatomical structures. For a 3D full body scan, if you had a control-bar over this 2d view you lot would starting time from a 2d piece of the head, and past increasing y'all would end up in the legs. Let'south practically call this view the "drone plane" or "tiptop-view". Below y'all tin encounter different slices of a brain MRI.

For anatomical consistency reasons, a slice well-nigh the head is referred to equally superior compared to a piece closer to the feet, which is called inferior.

Does it seem complex? Information technology is, I know! But it'southward important for your mental sanity if you want to survive in the field. Unfortunately, the other two planes assume dissimilar directions for positive 😢.

For completeness, and to present something a little bit more contempo below you run into an axial CT of a patient. Radiologists take highlighted irregularities that may be due to COVID. I am simply resharing the image hither to illustrate the axial piece, looking from the top to the patient. The particular slice refers in the lungs.

ct-scan-axial-view The 30-year-old woman had patches that resemble frosted glass, consistent with COVID-19 pneumonia.(Supplied: Radiopaedia, Dr Bahman Rasuli)

Sagittal airplane

Basically, this is a side view. Instead of looking from above the patient, now we wait from the side. The side can be either correct or left. Which side and direction is the positive i, depends on the coordinate organization! The sure thing is that from this view (plane) you tin come across the patient'south ear! As you move through this axis you meet the projected tissues like lungs, basic.

A sagittal view of a brain MRI can be illustrated beneath:

Coronal airplane

In this view, we traverse either by looking in the eyes (anterior plane) or past looking in the back of a patient (posterior plane). I promise y'all get the thought past at present.

In order to make certain that you will never forget what a coronal plane is, I constitute this awful halloween gif:

The highlighted terms in bold define the different Anatomical coordinate systems, every bit illustrated below.

$L P Southward=\left\{\begin{array}{c}\text { from right towards left } \\ \text { from anterior towards posterior } \\ \text { from junior towards superior }\finish{array}\right\}$

$R A Due south=\left\{\begin{array}{c}\text { from left towards right } \\ \text { from posterior towards anterior } \\ \text { from inferior towards superior }\end{assortment}\right\}$

Tip: LPS is used by DICOM images and by the ITK toolkit (simpleITK in python), while 3D Slicer and other medical software utilize RAS.

Medical Image coordinate system (Voxel space)

This is the part that comes more intuitively for people with a computer vision groundwork. If you have any feel with other 3D deep learning domains, I can assure you that this is the place that y'all will discover some rationality and relevant context, at terminal!

Medical modalities create 3D arrays/tensors of points and cells which get-go at the upper left corner, similar to an RGB photographic camera for 2nd. The i centrality increases to the correct (width), the j centrality to the bottom (peak), and the k axis backward (the 3rd similarly to the feature maps of a conv net).

In addition to the intensity value of each voxel (i j k), the origin (i.e. MRI) and spacing of the coordinates are stored in the meta-data of the medical prototype (either dicom tags, header file of nifty images, you lot proper noun it.. ).

Voxel spacing is basically the real size of the voxel that corresponds to a 3D region and the altitude between ii voxels.

This is pregnant if nosotros desire for example to measure the volume of a cancer tumor jail cell.

Note that, it is possible to resample to a bigger voxel size to reduce the size of the medical image. This tin can be understood as a downsampling operation of a signal. Medical people will say that in this way we continue the field of view the aforementioned, but permit's simply say that information technology'southward a kind of downsampling technique in the voxel space.

Note: if the voxels are of the aforementioned size in all 3 dimensions we call the image isotropic, similar to isotropic scaling in RGB images.

At present that nosotros see all these prissy coordinate theories, allow's see how we can manipulate and transform from 1 globe to another.

Moving betwixt worlds

In order to move from one world to another, we demand a magic key: information technology's called the well-known affine matrix. With this, nosotros can movement from one earth to another via a so-called affine transformation. Just what is an affine transformation?

Affine transformation

Before affine transformation let's analyze what a geometric transformation is. A geometric mapping/transformations is a fashion to clarify that the voxel intensity does not modify.

And then, based on the definition of Wikipedia for affine [7] : in geometry, an affine transformation is a geometric mapping of an affine infinite that preserves a lot of properties such as it sends points to points, lines to lines, planes to planes. Furthermore, it also preserves the ratio of the lengths of parallel lines. However, an affine transformation does not necessarily preserve angles between lines or distances betwixt points.

In math, to correspond translation and rotation together we demand to create a square affine matrix, which has one more dimensionality than our space. Since nosotros are in the 3D space we demand a 4D affine matrix in medical imaging. With the affine matrix, we tin correspond whatever linear geometrical transformation (translation, rotation), by a matrix multiplication, every bit illustrated below.

$\left(\begin{array}{c} x_{1} \\ x_{ii} \\ x_{iii} \\ 1 \end{array}\right)=\left(\begin{array}{cccc} A_{11} & A_{12} & A_{xiii} & t_{ane} \\ A_{21} & A_{22} & A_{23} & t_{2} \\ A_{31} & A_{32} & A_{33} & t_{3} \\ 0 & 0 & 0 & ane \stop{assortment}\right)\left(\begin{assortment}{c} i \\ j \\ g \\ 1 \cease{array}\right)$

To this cease, nosotros can go from the voxel infinite to world space coordinates of the imaging modality.

Annotation: in the affine transformation, elements indexed by A represent translation and t-indexed elements represent rotation.

Moving from one modality to another

We already saw that the affine is the transformation from the voxel to world coordinates. In fact, the affine was a pretty interesting property: the changed of the affine gives the mapping from earth to voxel. As a consequence, we can get from voxel infinite described by A of one medical epitome to another voxel infinite of another modality B. In this fashion, both medical images "live" in the same voxel space.

$\left[\begin{array}{c} i^{\prime} \\ j^{\prime} \\ k^{\prime} \\ 1 \end{array}\correct] = B^{-1} A\left[\begin{array}{c} i \\ j \\ chiliad \\ 1 \cease{array}\right]$

Let'southward see some compact lawmaking to perform this performance:

                          import                              scipy            
                                        def                                          transform_coordinate_space              (              modality_1              ,                              modality_2              )              :                          
                                                      """            
                                          Transfers coordinate space from modality_2 to modality_1            
                                          Input images are in nifty/nibabel format (.nii or .nii.gz)            
                                          """                          
                                          aff_t1                            =                              modality_1              .              affine            
                                          aff_t2                            =                              modality_2              .              affine            
                                          inv_af_2                            =                              np              .              linalg              .              inv              (              aff_t2              )                          
                                          out_shape                            =                              modality_1              .              get_fdata              (              )              .              shape            
                                          T                            =                              inv_af_2              .              dot              (              aff_t1              )                          
                                          transformed_img                            =                          
                          scipy              .              ndimage              .              affine_transform              (              modality_2              .              get_fdata              (              )              ,                              T              ,                              output_shape              =              out_shape              )                          
                                                      return                              transformed_img

Let'south meet what happens if we apply this code in two different MRI images:

medical-image-transformations-to-the-same-voxel-space Moving between voxel spaces

The first 3 slices are from the referenced image, the side by side 3 slices are from the image that nosotros want to transform, and the third one is the transformed image to the referenced world. They both live in the same affine world and nosotros tin can visualize them side by side and they also accept the same shape.

At present that we briefly covered some coordinate system concepts, permit's hash out the DICOM nightmare!

All modern medical imaging modalities like X-Rays, Ultrasound, CT (computed tomography), and MRI (Magnetic Resonance Imaging) support DICOM and use it extensively. DICOM is first of all an Interface Definition. It'southward success relies on the power to integrate medical modalities manufactured by unlike vendors. And unfortunately for the medical earth, this was near impossible before DICOM. For the tape, integrating medical equipment of different vendors used to be a huge consequence. Actually, this is the reason that DICOM has naturally become the manufacture standard.

DICOM is a unique format as it does not only store the medical epitome data but also data-sets, which are made up of attributes. For our readers with a software engineering science background let'due south simply say that information technology contains a huge amount of meta-data with the epitome (and usually redundant). The meta-data comprise critical information that must be kept within the file to ensure they are never separated from each other.

Tip: To summarize, the core of DICOM is both a file format and a networking protocol.

DICOM File Format: All medical images tin be saved in DICOM format. Medical imaging modalities create DICOM files. The huge adoption of DICOM files is justified because they contain more than just medical images, equally explained. Specifically, every DICOM file holds demographic patient information (name, birth date), too as important acquisition data (due east.g., type of modality used and its settings), and the context of the imaging study (i.e. radiotherapy series, patient history). As a automobile learning engineer, this is the most critical concept that you demand to deal with Dicom files.
DICOM Network Protocol: However, DICOM does not only define the image and data-sets (meta-information) just as well the transport layer protocol, more or less like TCP or whatever other protocol. The unabridged standard is fabricated up of multiple related but independent sections. Nevertheless, all medical imaging applications that are continued to the hospital network apply the DICOM protocol to commutation information, which is mainly DICOM images. Moreover, the DICOM network protocol is used to search for imaging studies in the archive and restore imaging studies to the workstation in club to display it. Finally, everything complex abstraction comes with a proficient part: Since DICOM is a complex protocol it allows for multiple commands such as schedule procedures, written report statuses.

Tip: In medical image analysis nosotros are mostly interested in understanding our data and then we can preprocess them to train a deep neural network.

From at present on, we will refer to DICOM as the file format for simplicity.

DICOM and deep learning

If you work on a project with DICOM data, you probably have to bank check a petty fleck of the standard. But allow's start with what nosotros would really want to have to apply our deep learning model. And then ideally, we would merely desire to do something like this for any modality:

                          import                              awesome_library            
                          medical_img_volume_in_numpy                            =                              awesome_library              .              loadDicom              (              path              )

This currently doesn't exist for DICOM because of the diversity of the different modalities and the unlike particularities that be in the Dicom images. For instance, a computed tomography image has different metadata (tags) than a magnetic resonance prototype.

Nevertheless, if you manage to convert your files in the nifty format, in that location is an crawly library called nibabel that does exactly what nosotros desire.

Tip: there is a magic library that reads peachy ( * .nii) information - NOT DICOM- in Python called nibabel that does what we wanted in the first identify.

We will detail below on that.

Fortunately, in that location are two awesome python packages that can salvage the states time: pyDICOM [5] and dcm2niix. With pyDICOM we can read and dispense Dicom files or folders. Some of you may wonder why this performance is not piddling! Is it so difficult to just get a 3D NumPy assortment of values?

The answer is aye, simply because normally Dicom files are usually organized every bit this: every unmarried 2d image piece is a different Dicom file. It is common that the dissimilar exams from multiple modalities are not fifty-fifty in the same folder. Existent-globe medical data are messy!

dcm2niix

On the other hand, the awesome tool called dcm2niix tin can catechumen a DICOM folder that contains a multi-sequence file in another format that fits our deep learning purposes chosen nifti. dcm2niix is designed to convert neuroimaging data from the DICOM format to the NIfTI format.

As an example you tin run this in an Ubuntu last:

$ sudo apt install dcm2niix

$ cd where_the_dicom_folder_of_all_the_slices_is

$ dcm2niix dicom_folder

Yous tin can sympathise how to utilize this past merely typing $ dcm2niix in your terminal:

Compression will be faster with 'pigz' installed

Chris Rorden's dcm2niiX version v1.0.20171215 (OpenJPEG build) GCC7.3.0 (64-bit Linux)

usage: dcm2niix [options] <in_folder>

Options :

-1..-ix : gz compression level (1=fastest..9=smallest, default half-dozen)

-b : BIDS sidecar (y/due north/o(o=merely: no NIfTI), default y)

-ba : anonymize BIDS (y/n, default y)

-c : comment stored as NIfTI aux_file (upwardly to 24 characters)

-d : improvidence volumes sorted by b-value (y/n, default n)

-f : filename (%a=antenna (gyre) number, %c=comments, %d=description, %e echo number, %f=folder name, %i ID of patient, %j seriesInstanceUID, %k studyInstanceUID, %thou=manufacturer, %northward=proper noun of patient, %p=protocol, %due south=series number, %t=time, %u=acquisition number, %v=vendor, %x=study ID; %z sequence proper noun; default '%f_%p_%t_%due south')

-h : show help

-i : ignore derived, localizer and 2D images (y/n, default n)

-thou : merge 2D slices from same series regardless of written report time, echo, gyre, orientation, etc. (y/northward, default n)

-northward : only convert this series number - can be used up to 16 times (default convert all)

-o : output directory (omit to save to input folder)

-p : Philips precise float (non display) scaling (y/due north, default y)

-due south : single file fashion, do not catechumen other images in folder (y/n, default n)

-t : text notes includes private patient details (y/n, default n)

-u : up-to-engagement check

-v : verbose (northward/y or 0/ane/2 [no, yes, logorrheic], default 0)

-x : crop (y/n, default n)

-z : gz compress images (y/i/n/3, default n) [y=pigz, i=internal:zlib, n=no, 3=no,3D]

Defaults file : /domicile/nikolas/.dcm2nii.ini

Examples :

dcm2niix /Users/chris/dir

dcm2niix -c "my comment" /Users/chris/dir

dcm2niix -o /users/cr/outdir/ -z y ~/dicomdir

dcm2niix -f %p_%s -b y -ba north ~/dicomdir

dcm2niix -f mystudy%s ~/dicomdir

dcm2niix -o "~/dir with spaces/dir" ~/dicomdir

I have used this for converting PET images, 4D CT, CT, and even cone-beam CT (CBCT). The catchy office comes when you lot want to read labels/annotations. Of course, in that location is not a single fashion to do this and it depends on the problem and the annotation toolbox. As soon as I understand this role improve, I will provide more than info on this. For the record, commonly the '-m y' selection is needed to merge the slices regardless of written report time for functional imaging. Functional means that the images are not structural, but they have a lot of timesteps.

Tip: Structural medical images are like photographic camera images (static), while functional medical images are kind of like videos.

The almost famous functional imaging is encephalon fMRI. As a result, a functional medical epitome is 4-dimensional. A beautiful style to understand this is by watching this video that assembly encephalon action based on encephalon fMRI signals with music.

You can read more than about such a written report about music and the listen by Meister et al. [6]

Reading .nii files in python with nibabel

If everything worked correctly you should now have a .nii file, too as a .json file that continents all the metadata that is not supported in the compact swell format. This tool is just amazing!

In general, nifty files finish in the suffix .nii or .nii.gz and is the data you lot probably download from deep learning challenges. As long as nosotros have this format nosotros can bask the ideal solution:

Thus, reading a nifty file, and getting the 3D volume in a numpy assortment is as simple as this:

                          import                              nibabel                            equally                              nib            
                          numpy_3D_medical_volume                            =                              bill              .              load              (              path              )              .              get_fdata              (              dtype              =              np              .              float32              )

Transform to RAS (canonical)

Merely nibabel library can do much more than this as nosotros volition see in the adjacent tutorial. Equally an instance, in order to associate the coordinate systems we will transform the dicom image to the RAS coordinate system. With nibabel, y'all tin can transform to canonical coordinates (RAS) like this:

                          import                              nibabel                            as                              nib            
                          img_nii                            =                              nib              .              load              (              path              )                          
                          img_nii                            =                              nib              .              as_closest_canonical              (              img_nii              )                          
                          img_np                            =                              img_nii              .              get_fdata              (              dtype              =              np              .              float32              )

Where to observe DICOM data

As a terminal note, I am providing these 2 links and then y'all tin play around with DICOM data:

Single exams to familiarize with loading files here
The cancer imaging annal for big-scale medical datasets
A lot of unlike radiotherapy dicom exams from Slicer github hither

Conclusion

Medical imaging has its weird counterparts, merely if you lot desire to solve interesting real-world wellness bug you have to take the time to sympathise your information. In this tutorial, we briefly introduced some concepts that will be in your everyday routine if you are going to work in a multi-disciplinary healthcare project. Don't only acquire your domain fundamentals; principal them! Motorcar learning includes the process of agreement our data. That's why I gently introduced a few loftier-level DICOM and medical image concepts from the perspective of an ML engineer. Finally, I would like to recommend the AI for medicine from coursera. Information technology offers multiple perspectives in AI for Medical Diagnosis, Medical Prognosis, and medical treatment. I just wish it existed earlier.

Cited equally:

@article { adaloglou2020dicomcoordinates ,

title = "Understanding coordinate systems and DICOM for deep learning medical prototype analysis" ,

writer = "Adaloglou, Nikolas" ,

periodical = "https://theaisummer.com/" ,

year = "2020" ,

url = "https://theaisummer.com/medical-image-coordinates/"

}

References

Coordinate systems and affines, nipy.org
Coordinate systems, 3D slicer documentation 3Dslicer.org
Introduction to DICOM, nipy.org
The DICOM standard
Official pyDICOM website
Meister IG, Krings T, Foltys H, et al. Playing piano in the mind--an fMRI study on music imagery and performance in pianists. Encephalon Res Cogn Brain Res. 2004;19(3):219-228. doi:10.1016/j.cogbrainres.2003.12.005
Wikipedia: affine transformation

* Disclosure: Please note that some of the links in a higher place might be affiliate links, and at no additional price to yous, we volition earn a commission if yous decide to make a buy after clicking through.

morrisgunfoop.blogspot.com

Source: https://theaisummer.com/medical-image-coordinates/