Workflows 🚦

In this section, we present practical examples of workflows that can be integrated into your systems, utilizing the data provided by our device's API.

We recognize that incorporating data from our API into your existing systems can be a complex task, particularly if this is your organization's first experience with such a device. These examples are designed to assist and guide you through this integration process.

Please note that the flexibility of our API allows for a wide range of workflow implementations. The examples here are not exhaustive and should not be interpreted as the only methods or as endorsements of particular practices. Each organization has unique needs and requirements. Consequently, it's crucial for managing organizations to develop workflows that align with their internal best practices and adhere to the guidelines set forth by their medical professionals.

Puzzle pieces

Consider the output from our device akin to pieces of a puzzle. The device generates a JSON file, abundant in data, organized under various keys. These keys serve as fundamental elements in your workflow design process. You can envision each key as a distinct branch in a decision-making tree or, more creatively, as a unique puzzle piece. This analogy highlights the flexibility and adaptability of the data; you can arrange and integrate these pieces in numerous configurations to suit your specific workflow needs.

Further reading

Please read the Output section of the Instructions For Use for more info.

There are more puzzle pieces

The API returns more than 100 rows of data, and these examples are only using 6 keys from the output. This means that there are way more puzzle pieces, that may be more useful to your organisation. Feel free to reach out to us to ask about what endpoints may be suited to your use case.

Arranging the puzzle

Let's delve into some practical examples showcasing how you can assemble the puzzle pieces to formulate an effective workflow.

Primary vs. secondary care

In this scenario, we demonstrate a workflow that bifurcates into two distinct paths: Primary Care and Secondary Care. The decision between these outcomes is illustrated in the following table:

Primary care	1
Secondary care	2

Building upon this, a potential workflow might incorporate specific puzzle pieces that assist an organization in refining and enhancing the efficiency of its referral process:

Thresholding

To determine the thresholds, the organisation must take into account two things:

1. What are the possible outcomes of the workflow?
2. What is the sensitivity and specificity of the parameter?

In this case, the outcome of the flowchart always includes seeing a healthcare practitioner. Due to this, if the organisation is looking to reduce patient waiting lists to increase patient safety, it may be a good idea to establish a high threshold.

You can find more information on this topic in the Thresholding section, where we explain the confusion matrix of some parameters.

Remote vs. in-person consultation

An organization may aim to enhance its consultation process by efficiently determining the most suitable method for each case, be it remote or in-person consultation.

For this example, we continue with the two established outcomes: Primary Care and Secondary Care. We expand this decision-making process by adding the dimension of consultation type: Remote or In-Person, as depicted in the following table:

	🤳 Remote	🏥🚶 In-person
Primary care	R1	P1
Secondary care	R2	P2

Why this matters

Implementing a strategy to improve the efficiency of consultations and minimize patient wait times could involve directing more cases to primary care physicians or leveraging quicker consultation methods. The table below shows the desired direction for each care level and consultation type:

	🤳 Remote	🏥🚶 In-person
Primary care	Goal: Increase ↑	-
Secondary care	-	Goal: Decrease ↓

Workflow example

The workflow designed to achieve these objectives could be structured as follows:

This can be complemented with condition-specific questionnaires such as the 7-point checklist (7PC). The 7-point checklist (7PCL) has been recommended by NICE (2005) for routine use in UK general practice to identify clinically significant lesions which require urgent referral.

In the following flowchart, the output of the IsMalignantSuspicion is followed by the 7PC questionnaire, to further specify the right method for consultations.

Thresholding

To determine the thresholds, the organisation must take into account two things:

1. What are the possible outcomes of the workflow?
2. What is the sensitivity and specificity of the parameter?

In this case, the outcome of the flowchart always includes seeing a healthcare practitioner. Due to this, if the organisation is looking to reduce patient waiting lists to increase patient safety, it may be a good idea to establish a high threshold.

You can find more information on this topic in the Thresholding section, where we explain the confusion matrix of some parameters.

Refining communication methods for consultation

To further optimize the consultation process, we can specify different methods for both remote and in-person consultations, aligning them with specific objectives:

	🤳 Remote		🏥🚶 In-person
	Chat consult	Video consult	Internal team	External team
Primary care	Goal: Increase ↑	-	-	-
Secondary care	-	-	Goal: Decrease ↓	Goal: Decrease ↓

Chronic patient follow-up

Organizations may aim to enhance remote monitoring of chronic patients to improve patient oversight and evaluate treatment effectiveness.

In this instance, we continue to differentiate between Primary Care and Secondary Care. However, we now factor in the status of the patient's condition, categorized as either Progressing as Expected or displaying an Anomaly, as illustrated below:

Progressing as expected	OK
Anomaly	!

While the device does not explicitly state if a patient's condition is evolving as expected, it does provide a scoreCategorySeverity score. This score, comprising values 0, 1, 2, and 3, indicates the level of condition impact, with 2 and 3 representing moderate or severe conditions.

Further reading

Please read the Output section of the Instructions For Use for more info.

Based on this, an organization might implement the following workflow:

Thresholding

As you can see, the puzzle pieces sometimes require deciding on a threshold from which different actions may be taken. Different use cases may require different thresholds, usually depending on the possible outputs of the workflow.

How to decide a threshold

To measure what is the appropriate threshold for a parameter, we use confusion matrixes.

A confusion matrix is a fundamental tool in statistical classification and machine learning. It is a specific table layout that allows visualization of the performance of an algorithm, typically a classifier. The matrix compares the actual target values with those predicted by the model, providing insight into not only the performance of the classifier but also the types of errors it is making.

The confusion matrix is typically a 2x2 matrix for binary classification tasks:

• True Positives (TP): These are cases where the classifier correctly predicts the positive class.
• True Negatives (TN): These are cases where the classifier correctly predicts the negative class.
• False Positives (FP): These are cases where the classifier incorrectly predicts the positive class (also known as Type I error).
• False Negatives (FN): These are cases where the classifier incorrectly predicts the negative class (also known as Type II error).

In a classification task, especially in probabilistic classifiers, the threshold is a concept that must be correctly understood. In essence: a threshold is the point at which the probability of a data point being in one class over another is decided.

• When the model's predicted probability for a class is higher than the threshold, it classifies the data point into that class.
• Conversely, if the probability is lower than the threshold, it classifies the data point into the other class.

Where is this value?

The confusion matrix is provided by us, as it is the result of the extensive testing and validation of the device. This value is usually expressed in terms of specificity and sensibility. May vary from one task to another. For instance, the image quality has a different value than the classification of a condition.

With the information of sensitivity and specificity, each organisation may decide on a specific threshold, depending on their use case.

Adapting the threshold to the use case

Determining the appropriate threshold is fully dependent on the use case. For example, in deciding whether to refer to a specialist or a primary care practitioner:

• A lower threshold may lead to more cases being referred to a specialist, potentially catching more true positive cases but also increasing false positives.
• A higher threshold may reduce the number of false positives but can lead to missing true positive cases.

Therefore, the threshold must be chosen by the organisation with its specific use case in mind, balancing the outcomes of false positives and false negatives.

Threshold for isMalignantSuspicion

isMalignantSuspicion is one of the values output by the device for which an organisation may need to choose a threshold.

isMalignantSuspicion represents a quantified suspicion level: the higher the value, the greater the suspicion of malignancy. Unlike a binary output, this continuous scale provides a more nuanced understanding but also necessitates a well-defined threshold to classify a case as malignant or not.

isMalignantSuspicion is not a straightforward binary indicator but a cumulative measure derived from summing the probabilities of various classes deemed malignant by a multi-class classification algorithm. This approach is significant for several reasons:

1. Ambiguous nature of lesions: lesions can exhibit characteristics that span multiple classes. In such cases, a singular classification might be insufficient to capture the full scope of the lesion's nature.
2. Evolving conditions: Some lesions represent conditions in transition; for example, a benign lesion that is in the process of becoming malignant.

That is why capturing the probabilities across classes allows for a more dynamic understanding of the lesion's current state.

Practical example

Imagine 100 people ask for a consultation, but only 12 of them actually have a malignant condition. If we refer or prioritise cases where isMalignantSuspicion is higher or equal to 05101520253035404550556065707580859095100, then...

12 🙌 Are prioritised and should be prioritised True positives (TP)	88 😅 Are prioritised but should not be prioritised False positives (FP)
0 😣 Are not prioritised but should be prioritised False negatives (FN)	0 👏 Are not prioritised and should not be prioritised True negatives (TN)

Keep in mind that the number 12 in every 100 people does not give insight as to how many people actually will submit malignant images. What this gives you is knowledge of the accuracy of the algorithm, with a given ratio of malignant to non-malignant images.

Threshold for mediaValidity.score

mediaValidity is one of the values output by the device for which an organisation may need to choose a threshold.

mediaValidity contains different aspects of the quality of an image. Inside this key, there are several keys such as isValid, hasEnoughQuality, isDermatologyDomain and most importantly, score.

Unlike a binary output, score provides a continuous scale with a more nuanced understanding of the image quality. However, it necessitates a well-defined threshold to classify a case as having enough quality or not.

Practical example

Imagine 100 images are sent to the device. Each image has a mediaValidity.score between 1 and 100. In order to separate good quality images from bad quality images, any image with a mediaValidity.score higher than 0%10%20%30%40%50%60%70%80%90%100% will be accepted.

In this case, due to the threshold of 0%10%20%30%40%50%60%70%80%90%100%, out of the 100 images, only 87 have enough image quality. As a result:

84 📸👌 Have enough quality and are proccessed True positives (TP)	6 😅📸 Have enough quality but are discarded False positives (FP)
3 😅📸 Don't have enough quality but are processed False negatives (FN)	8 📸👏 Don't have enough quality and are discarded True negatives (TN)

This can also be understood through an acceptance chart. In the following chart, you will see how many images are accepted depending on the threshold of the mediaValidity.score. As examples, there are marks set at the thresholds for 40 and 80. In the second case, one can see that half of the images will be rejected.

40 → Acceptance: 99.16%

80 → Acceptance: 49.02%

Keep in mind that this does not give insight as to how many bad images will users actually send. What this gives you is knowledge of the accuracy of the image quality algorithm. It explains the confusion matrix of the DIQA algorithm, with a given ratio of good to bad images.

Where does this come from?

To determine and verify the accuracy of the algorithm, we have tested the algorithm with a dataset that has been labelled by experts at this task. That is how we derive the confusion matrix.

Further reading: Dermatology Image Quality Assessment (DIQA): Artificial intelligence to ensure the clinical utility of images for remote consultations and clinical trials. Journal of the American Academy of Dermatology.

Prioritisation

Organizations can create workflows to enhance the prioritization of patient care.

Each organization develops its own protocols for assigning priority to patients. The goal is to optimize referral processes, ensuring that patients receive timely and appropriate care from the right healthcare professionals.

For educational purposes, this section offers examples of potential workflows that a managing organization might employ. These are based on the diagnostic report information provided by our API, to effectively prioritize patient episodes.

Malignancy

One critical factor in healthcare decision-making is the evaluation of malignancy risk.

The API response includes a segment labeled preliminaryFindings, which can present information relevant to this assessment. This data can be instrumental in informing the prioritization process, as detailed below:

Preliminary findings

{
  // ...
  "preliminaryFindings": {
    // ...
    "isMalignantSuspicion": 62
    // ...
  }
  // ...
}

The parameter isMalignantSuspicion may be used to help in the decision-making of assigning priority to patients.

Here's a straightforward rule to categorize patient priority based on this parameter:

• When isMalignantSuspicion is from 0 to 30%, the patient can be categorized as normal priority.
• If isMalignantSuspicion exceeds 30%, the patient should be considered high priority.

This decision-making process can be visualized using a flowchart:

Expanding upon this:

• Patients with isMalignantSuspicion exceeding 30% are not only a high priority but may also be candidates for fast-tracking.

Another flowchart can help illustrate this extended decision-making criterion:

Order by malignancy

Organizations can refine their prioritization strategies by ordering cases according to the level of malignancy suspicion. For example, two patients who both fall into the High-priority category due to their malignancy suspicion levels — one at 60 and another at 30 — can be further triaged. This approach involves ranking patients within the High-priority group from highest to lowest based on their malignancy suspicion score.

This method enables more nuanced prioritization, ensuring that those with higher suspected risks are attended to more urgently.

If the managing organization opts to make this malignancy suspicion information visible to healthcare professionals (HCPs), the following type of chart could be implemented to facilitate this refined prioritization process:

isMalignantSuspicion: 6%	isMalignantSuspicion: 62%

Other workflows

Severity

The severity of a condition is one of the variables that HCPs use in their decision-making.

A response from the API contains a section called scoringSystems that may include the following information:

scoringSystems

{
  // ...
  "scoringSystems": [
    {
      "scoringSystem": {
        "name": "Acne lesion estimation grading index",
        "code": "ALEGI"
      },
      "score": 8,
      "scoreCategorySeverity": 1
      // ...
      "scoreCategories": [
        {
          "code": "None",
          "category": "None",
          "min": 0.0,
          "max": 0.0,
          "severity": 1,
          "severityAsString": "low"
        },
        {
          "code": "Grade 1",
          "category": "Grade 1",
          "min": 0.0,
          "max": 10.0,
          "severity": 1,
          "severityAsString": "low"
        },
        // ...
      ]
    }
  ]
}

As you can see, the scoringSystem object includes a score value. In that example, we see that the severity score is 8.

Then, in the array called scoreCategories, you can see that the values between 1 and 10 fall under the severity 1 - meaning that it is the lowest severity possible. And the category for that severity is Grade 1, which is a semantic interpretation of the score that directly reflects the severity.

A very simple rule would be:

• If the scoreCategorySeverity is 1, it would trigger workflow A
• If the scoreCategorySeverity is 3, it would trigger workflow B

This can be useful, for example, when an HCP must decide whether or not to refer a patient to a specialist. It can also be useful information to monitor de effectiveness of a treatment.

In this example, workflow A would be the one applicable when the severity of the condition is under control, for instance, because it is correctly responding to the treatment,

Furthermore, you can enrich the process by adding information on the patient's history. For instance, if you also send data inside the key previousMedia when you post your request, the result could be:

• If the scoreCategorySeverity is 2, and the previous photo was not 3, it would trigger workflow C
• Otherwise, it would fallback to workflow A

The managing organisation may also decide to show this information to the HCP, in which case a chart such as the following could be implemented:

The score for the severity can be displayed visually. The way of doing this is using the values in the array ranges:

scoreCategorySeverity: 1	scoreCategorySeverity: 3

Prior existence of the condition

Whether a condition is new for a patient, or already diagnosed to that patient, is one of the variables that HCPs use in their decision-making.

A response from the API contains a section called conclusions that may include the following information:

Conclusions

{
  // ...
  "conclusions": [
    {
      "code": {
        "code": "DA04.5",
        "codeSystem": "ICD-11"
      },
      "name": "Mucous cyst",
      "probability": 87.77
    }
    // ...
  ]
  // ...
}

What that information shows is that a condition has a probability of 87.77. The name of the condition is Mucous cyst, coded as DA04.5 according to the system ICD-11.

What an HCP could do is look up whether or not that condition has been previously diagnosed to the patient, and use that information in their decision-making.

Pre-malignancy

You may have seen that the API returns a value named isPreMalignantSuspicion, which is different to isMalignantSuspicion, as follows:

Preliminary findings

{
  // ...
  "preliminaryFindings": {
    // ...
    "isMalignantSuspicion": 0,
    "isPreMalignantSuspicion": 8.6000000000000001
    // ...
  }
  // ...
}

A premalignant condition (also known as a precancerous condition) is one that if left untreated may turn into cancer.

Out of the possible conditions output by the API, there are

• 10 malignant conditions
• 42 pre-malignant conditions

This means that is more likely that a report may return a higher pre-malignancy suspicion.

Here is a sample table of conditions classified as malignant or pre-malignant, according to the relevant literature:

	Condition	Malignant	Pre-malignant
1	Acanthosis nigricans
2	Acne
3	Acne fulminans
4	Acrolentiginous melanoma
5	Actinic cheilitis
6	Actinic elastosis
7	Actinic keratosis
8	Actinic porokeratosis
+ +	+ + + + + + + + + + + + + +	+ + + + + + +	+ + + + + + + +
234	Xeroderma pigmentosum
235	Xerosis

The complete table is 235 rows long. Feel free to reach out to your sales representative to request further information.

caution

Keep in mind that the conclusions key of the diagnostic report returns a full list of the conditions detected by the device. As such, an organisation may choose to assign malignancy of premalignancy differently to the value shown in isMalignantSuspicion and isPreMalignantSuspicion. The device provides those values for the convenience of the organisation and adds more transparency into the inner workings of the device.

The steps that an HCP may follow are: