Browse AI research papers authored by Liam Hazan. Found 3 papers.

arXiv:2509.04519v1 Announce Type: new Abstract: Extracting structured clinical information from radiology reports is challenging, especially in low-resource languages. This is pronounced in Crohn's disease, with sparsely represented multi-organ findings. We developed Hierarchical Structured Matching Prediction BERT (HSMP-BERT), a prompt-based model for extraction from Hebrew radiology text. In an administrative database study, we analyzed 9,683 reports from Crohn's patients imaged 2010-2023 across Israeli providers. A subset of 512 reports was radiologist-annotated for findings across six gastrointestinal organs and 15 pathologies, yielding 90 structured labels per subject. Multilabel-stratified split (66% train+validation; 33% test), preserving label prevalence. Performance was evaluated with accuracy, F1, Cohen's $kappa$, AUC, PPV, NPV, and recall. On 24 organ-finding combinations with $>$15 positives, HSMP-BERT achieved mean F1 0.83$pm$0.08 and $kappa$ 0.65$pm$0.17, outperforming the SMP zero-shot baseline (F1 0.49$pm$0.07, $kappa$ 0.06$pm$0.07) and standard fine-tuning (F1 0.30$pm$0.27, $kappa$ 0.27$pm$0.34; paired t-test $p < 10^{-7}$). Hierarchical inference cuts runtime 5.1$times$ vs. traditional inference. Applied to all reports, it revealed associations among ileal wall thickening, stenosis, and pre-stenotic dilatation, plus age- and sex-specific trends in inflammatory findings. HSMP-BERT offers a scalable solution for structured extraction in radiology, enabling population-level analysis of Crohn's disease and demonstrating AI's potential in low-resource settings.

## Overview

- Research tackles extracting structured medical data from Hebrew radiology reports
- Uses advanced machine learning technique called **Hierarchical Section Matching Prediction (HSMP)**
- Focuses specifically on [Crohn's disease](https://aimodels.fyi/health/crohns-disease) diagnostic documentation
- Develops a sophisticated text analysis approach for medical record processing

## Plain English Explanation

Medical professionals often struggle to convert complex, free-text radiology reports into structured, easily analyzable data. This research addresses that challenge by developing a smart AI system that can automatically parse Hebrew-language medical documents.

Imagine trying to read hundreds of handwritten medical reports and systematically categorizing every detail - it would be incredibly time-consuming. The **HSMP BERT** model acts like an intelligent assistant that can rapidly scan documents, understand context, and extract precise information about [Crohn's disease](https://aimodels.fyi/health/crohns-disease) findings.

The system works by breaking down medical reports into hierarchical sections, teaching the AI to recognize and match specific medical terminology and structural patterns unique to radiology documentation.

## Key Findings

- Successfully developed a **machine learning model** capable of extracting structured data from Hebrew medical texts
- Demonstrated high accuracy in identifying and categorizing medical information
- Proved the potential of [advanced natural language processing](https://aimodels.fyi/ai/natural-language-processing) techniques in medical documentation

## Technical Explanation

The researchers leveraged **BERT** (Bidirectional Encoder Representations from Transformers) architecture, a state-of-the-art [machine learning approach](https://aimodels.fyi/ai/machine-learning) for understanding language context. By implementing a hierarchical section matching prediction strategy, they created a model that:

- Understands document structure
- Recognizes contextual medical terminology
- Extracts precise, structured information from unstructured text

The approach involves training the model on a dataset of Hebrew radiology reports, teaching it to recognize patterns specific to [Crohn's disease](https://aimodels.fyi/health/crohns-disease) documentation.

## Critical Analysis

While promising, the research has several limitations:
- Tested only on Hebrew-language documents
- Focused solely on [Crohn's disease](https://aimodels.fyi/health/crohns-disease)
- Requires significant computational resources
- Needs extensive validation across different medical contexts

The model's performance might vary with different medical specialties or languages, suggesting further research is necessary.

## Conclusion

This research represents a significant step in automating medical information extraction. By demonstrating the potential of [advanced natural language processing](https://aimodels.fyi/ai/natural-language-processing) techniques, the study opens new possibilities for efficient, accurate medical documentation across languages and specialties.

The **HSMP BERT** approach could revolutionize how medical professionals process and analyze complex diagnostic reports, potentially saving time and improving diagnostic accuracy.

Hierarchical Section Matching Prediction (HSMP) BERT for Fine-Grained Extraction of Structured Data from Hebrew Free-Text Radiology Reports in Crohn's Disease

Drug discovery typically consists of multiple steps, including identifying a target protein key to a disease's etiology, validating that interacting with this target could prevent symptoms or cure the disease, discovering a small molecule or biologic therapeutic to interact with it, and optimizing the candidate molecule through a complex landscape of required properties. Drug discovery related tasks often involve prediction and generation while considering multiple entities that potentially interact, which poses a challenge for typical AI models. For this purpose we present MAMMAL - Molecular Aligned Multi-Modal Architecture and Language - a method that we applied to create a versatile multi-task multi-align foundation model that learns from large-scale biological datasets (2 billion samples) across diverse modalities, including proteins, small molecules, and genes. We introduce a prompt syntax that supports a wide range of classification, regression, and generation tasks. It allows combining different modalities and entity types as inputs and/or outputs. Our model handles combinations of tokens and scalars and enables the generation of small molecules and proteins, property prediction, and transcriptomic lab test predictions. We evaluated the model on 11 diverse downstream tasks spanning different steps within a typical drug discovery pipeline, where it reaches new SOTA in 9 tasks and is comparable to SOTA in 2 tasks. This performance is achieved while using a unified architecture serving all tasks, in contrast to the original SOTA performance achieved using tailored architectures.
  The model code and pretrained weights are publicly available at https://github.com/BiomedSciAI/biomed-multi-alignment and https://huggingface.co/ibm/biomed.omics.bl.sm.ma-ted-458m.

## Overview

- MAMMAL is a new multi-modal architecture that aligns language and molecular data.
- It enables tasks like generating molecule descriptions from images, or predicting molecular properties from text.
- The architecture uses self-supervised pretraining on large datasets to learn rich representations.

## Plain English Explanation

MAMMAL is a new AI system that can understand and work with both language and molecular data. It allows you to do things like [describe a molecule from an image](https://aimodels.fyi/papers/arxiv/molbind-multimodal-alignment-language-molecules-proteins), or [predict a molecule's properties from a text description](https://aimodels.fyi/papers/arxiv/y-mol-multiscale-biomedical-knowledge-guided-large).

The key idea is to train the system on lots of examples of language and molecules together, so it can learn how the two are connected. This allows it to build rich representations that capture the relationships between text, images, and molecular structures.

For example, the system might learn that certain words and phrases are associated with particular molecular shapes or chemical properties. It can then use this knowledge to generate relevant text when shown a molecular image, or predict molecular attributes from a text description.

This multi-modal approach is powerful because it allows the AI to leverage insights and information across different modalities, rather than being limited to just one type of data. It opens up new possibilities for tasks like drug discovery, materials science, and other areas where both language and molecular information are important.

## Key Findings

- The MAMMAL architecture outperforms previous multi-modal models on a range of language-molecule alignment tasks.
- Pre-training MAMMAL on large datasets of text, images, and molecular data is crucial for its strong performance.
- MAMMAL can effectively transfer its learned representations to downstream tasks like molecule generation and property prediction.

## Technical Explanation

The core of the MAMMAL architecture is a multi-modal transformer model that takes as input text, molecular graph images, and molecular SMILES strings. Through self-supervised pre-training on large datasets, the model learns to align the representations of these different modalities.

The pre-training tasks include [masked language modeling](https://aimodels.fyi/papers/arxiv/feedback-aligned-mixed-llms-machine-language-molecule), [molecular graph reconstruction](https://aimodels.fyi/papers/arxiv/mammothmoda-multi-modal-large-language-model), and [cross-modal retrieval](https://aimodels.fyi/papers/arxiv/y-mol-multiscale-biomedical-knowledge-guided-large). This teaches the model to capture the semantic and structural relationships between language, images, and molecular structures.

The pre-trained MAMMAL model can then be fine-tuned for various downstream tasks, like generating molecule descriptions from images, or predicting molecular properties from text. The model's ability to effectively transfer its learned representations is a key strength.

## Critical Analysis

The MAMMAL paper provides a thorough evaluation, demonstrating strong performance on a diverse set of language-molecule alignment benchmarks. However, the authors acknowledge some limitations:

- The model's performance is still constrained by the quality and coverage of the pre-training data. Expanding the datasets could further improve results.
- The architecture is complex, with many components and hyperparameters to tune. Simplifying the design while maintaining performance is an area for future work.
- The paper does not deeply explore the model's reasoning or provide much insight into the learned representations. More interpretability analysis could be valuable.

Overall, MAMMAL represents an exciting advance in multi-modal AI for molecular applications. But as with any new technology, there is room for continued refinement and deeper understanding of its capabilities and limitations.

## Conclusion

The MAMMAL architecture is a powerful new tool for aligning language and molecular data. By learning rich cross-modal representations through large-scale pre-training, MAMMAL enables a wide range of language-molecule tasks with state-of-the-art performance.

This multi-modal approach opens up new possibilities for applications like drug discovery, materials design, and scientific communication. As the field of AI continues to evolve, we can expect to see increasingly capable systems that can fluidly combine and reason about different types of information - ultimately leading to transformative advances across science and technology.

MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language

Automatic conversion of free-text radiology reports into structured data using Natural Language Processing (NLP) techniques is crucial for analyzing diseases on a large scale. While effective for tasks in widely spoken languages like English, generative large language models (LLMs) typically underperform with less common languages and can pose potential risks to patient privacy. Fine-tuning local NLP models is hindered by the skewed nature of real-world medical datasets, where rare findings represent a significant data imbalance. We introduce SMP-BERT, a novel prompt learning method that leverages the structured nature of reports to overcome these challenges. In our studies involving a substantial collection of Crohn's disease radiology reports in Hebrew (over 8,000 patients and 10,000 reports), SMP-BERT greatly surpassed traditional fine-tuning methods in performance, notably in detecting infrequent conditions (AUC: 0.99 vs 0.94, F1: 0.84 vs 0.34). SMP-BERT empowers more accurate AI diagnostics available for low-resource languages.

## Overview

- This paper explores the use of prompt-learning techniques for structured information extraction from Crohn's disease radiology reports in a low-resource language.
- The researchers develop a novel approach that leverages pre-trained language models and customized prompts to extract relevant medical information from radiology reports, even in languages with limited available data.
- The proposed method demonstrates promising results for improving the accuracy and efficiency of information extraction in this clinical domain, potentially aiding clinicians and researchers working with Crohn's disease patients.

## Plain English Explanation

In this study, the researchers looked at a way to help doctors and scientists better understand medical information from X-ray and imaging reports about a disease called Crohn's disease. Crohn's disease is a condition that can cause inflammation and other problems in the digestive system.

The researchers developed a new technique that uses [pre-trained language models](https://aimodels.fyi/papers/arxiv/general-purpose-vs-domain-adapted-large-language) and customized "prompts" to extract important details from radiology reports, even in languages where there is not a lot of existing data available. Prompts are short instructions that help the language model focus on the right information.

This approach was tested on Crohn's disease radiology reports and showed promising results for improving the accuracy and efficiency of extracting relevant medical details. This could be very useful for clinicians treating Crohn's patients and researchers studying the disease, especially in areas where the local language has limited medical data available.

## Technical Explanation

The paper presents a novel approach for structured information extraction from Crohn's disease radiology reports in a low-resource language, leveraging the power of [prompt-learning](https://aimodels.fyi/papers/arxiv/towards-efficient-patient-recruitment-clinical-trials-application).

The researchers first fine-tune a pre-trained language model on a small corpus of Crohn's disease radiology reports in the target low-resource language. They then design customized prompts that guide the model to extract specific medical entities and attributes related to the Crohn's disease diagnosis, such as the location and severity of intestinal inflammation.

To evaluate their method, the team collects a dataset of Crohn's disease radiology reports in the low-resource language and annotates them with ground-truth structured information. They compare the performance of their prompt-based approach to other information extraction techniques, including [named entity recognition](https://aimodels.fyi/papers/arxiv/llms-biomedicine-study-clinical-named-entity-recognition) and [radiology report generation](https://aimodels.fyi/papers/arxiv/mrscore-evaluating-radiology-report-generation-llm-based) models.

The results demonstrate that the prompt-learning approach outperforms the baseline methods, achieving higher accuracy in extracting the relevant medical information from the Crohn's disease radiology reports. The researchers also conduct ablation studies to understand the impact of different prompt design choices on the model's performance.

## Critical Analysis

The paper presents a thoughtful and well-designed approach to addressing the challenge of structured information extraction in a low-resource language setting. The use of prompt-learning techniques is a promising direction, as it allows the model to focus on the specific medical entities and attributes relevant to Crohn's disease without requiring a large annotated dataset.

However, the paper does not extensively discuss the limitations of the proposed method. For example, it would be interesting to understand how the approach would scale to more diverse or noisy radiology report data, or how it might perform on other types of medical documents beyond radiology reports.

Additionally, the paper could have provided more insight into the potential biases or errors that might arise from the prompt-learning approach, and how these could be mitigated. A more thorough discussion of the ethical considerations, such as the potential misuse of the extracted information or the impact on patient privacy, would also strengthen the critical analysis.

Overall, this research represents an important step forward in leveraging [advanced language models](https://aimodels.fyi/papers/arxiv/serpent-vlm-self-refining-radiology-report-generation) for clinical data extraction in low-resource settings. Further exploration of the limitations and potential risks of the approach, as well as its broader applicability, would make for a more comprehensive understanding of the technique and its implications.

## Conclusion

This paper introduces a novel prompt-learning based approach for structured information extraction from Crohn's disease radiology reports in a low-resource language. The researchers demonstrate that their method outperforms baseline techniques, highlighting the potential of this approach to aid clinicians and researchers working with Crohn's disease patients, especially in regions with limited medical data availability.

While the paper presents a well-designed and promising solution, further research is needed to fully understand the limitations, potential biases, and broader applicability of the prompt-learning technique. Nonetheless, this work represents an important contribution to the field of clinical natural language processing and information extraction, with implications for improving patient care and advancing medical research in low-resource settings.

Liam Hazan

Authored papers

Can AI precisely map 90 specific Crohn's findings from Hebrew reports?

Could a single AI model master the language of molecules and modalities to accelerate drug discovery?

Can prompt-learning unlock the secrets hidden in Crohn's disease radiology reports, even when the language is rare and data is scarce?

Claim this author profile