This project aims to develop specialized generalist models in the field of biomedicine. These models are designed to excel at answering questions related to exams, clinical scenarios, and research problems while maintaining a broad general knowledge base to effectively handle cross-cutting fields.
To achieve this goal, we have constructed a large-scale, high-quality dataset of biomedical instructions mixing synthetic and manual data along with preference annotation, called UltraMedical. This dataset is built on the principles of diversity and complexity, ensuring that the models trained on it can handle a wide range of tasks and scenarios.
Our training process involves the use of advanced alignment technologies, including Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Odds Ratio Preference Optimization (ORPO). By leveraging these techniques and training large language models on the UltraMedical dataset, we aim to create powerful and versatile models that can effectively serve the needs of the biomedical community.
The UltraMedical Collections
The UltraMedical Collections is a large-scale, high-quality dataset of biomedical instructions, comprising 410,000 synthetic and manually curated samples.
Statistics of datasets in the UltraMedical collections is shown in following table, where datasets marked with ★ represent our customized synthetic data, while the others are adapted from publicly available data. # Filtered represents the remaining data size after filtering by model-based scoring, while # Instructions refers to the original size of the dataset.
| Category | Synthetic | Dataset | # Instructions | Avg.Len of Instruction | Avg.Score of Instruction | # Filtered |
|---|---|---|---|---|---|---|
| Medical Exam | ✘ | MedQA | 10.2k | 128.94 ± 44.4 | 7.35 ± 0.98 | 9.3k |
| ✘ | MedMCQA | 183k | 23.12 ± 15.44 | 4.73 ± 2.14 | 59k | |
| ✔︎ | ★ MedQA-Evol | 51.8k | 76.52 ± 24.97 | 8.07 ± 0.9 | 51.8k | |
| ✔︎ | ★ TextBookQA | 91.7k | 75.92 ± 25.77 | 7.72 ± 0.79 | 91.7k | |
| Literature | ✘ | PubMedQA | 211k | 218.2 ± 51.01 | 7.95 ± 1.08 | 88.7k |
| Open-ended | ✘ | ChatDoctor | 100k | 98.93 ± 50.81 | 6.83 ± 2.16 | 31.1k |
| ✘ | MedQuad | 47k | 8.21 ± 2.38 | 4.54 ± 2.43 | 6k | |
| ✔︎ | MedInstruct-52k | 52k | 36.05 ± 22.96 | 5.25 ± 2.16 | 23k | |
| ✔︎ | Medical-Instruction-120k | 120k | 84.93 ± 50.85 | 5.36 ± 3.18 | 25k | |
| ✔︎ | ★ WikiInstruct | 23k | 46.73 ± 11.1 | 8.8 ± 0.52 | 23k | |
| Mixed | Mixed | ☆ UltraMedical | 410k | 101.63 ± 79.39 | 8.2 ± 0.96 | 410k |
Construction
- Principle of Diversity
- UltraMedical encompasses a variety of question types, including medical exam questions, literature-based questions, and open-ended instructions (clinical questions, research questions, and others). It comprises 12 manual and synthetic datasets. For publicly available datasets, we have gathered questions from multiple sources, including medical exams, medical literature, clinical questions, and open-ended instructions. These datasets feature not only manually curated instructions but also prompted instructions from GPT-4. The various data sources preliminarily enable the diversity principle of the UltraMedical dataset.
- In addition to public datasets, we have created three synthetic datasets to augment the UltraMedical collection. One such dataset, named TextBookQA, consists of multiple-choice questions derived from medical textbooks, using questions from MedQA as in-context examples. The other, WikiInstruct, aggregates thousands of biomedical concepts from Wikipedia pages and expands them into more detailed knowledge and instructions.
- Principle of Complexity
- Beyond the diversity characteristic, UltraMedical also upholds the principle of complexity to inject knowledge and enhance reasoning through complex instructions. There are primarily two methods to enhance the complexity of instructions, either pre-hoc or post-hoc. The former involves starting with various seed instructions to synthesize new instructions, followed by employing self-evolution on these synthetic instructions. The latter method involves filtering instructions using heuristic rules or model-based rankers to select the most complex instructions.
- During the construction of the UltraMedical dataset, we employ both pre-hoc and post-hoc methods to enhance the complexity of the instructions. For publicly available datasets, we use gpt-3.5-turbo to assign a scale score ranging from 1 to 10 to each instruction, where 1 indicates an instruction that is easy to answer and 10 denotes one that is challenging for a powerful AI assistant. For our synthetic dataset, we combine pre-hoc and post-hoc methods to ensure the complexity of the instructions. Initially, we implement a two-step self-evolution process on all synthetic instructions, and then further filter them based on model-derived scores.
Annotation and Decontamination
We annotate answers using gpt-4-turbo to optimize responses for supervised fine-tuning. For multiple-choice questions, the chain-of-thought (CoT) method has proven effective in distilling knowledge from large to smaller language models. Therefore, we instruct gpt-4-turbo to answer each question step by step. Subsequently, we verify the answers against the ground truth and filter out incorrect responses. For incorrect answers, we further engage gpt-4-turbo with dynamically retrieved few-shot CoT examples from our annotated database. This process enables us to maximize the number of potential candidate samples while ensuring the quality of the completions.
To prevent test set leakage as a result of employing large-scale synthetic data, we conduct decontamination operations, similar to the methods outlined in the bagel project.
Medical imaging has always been a cornerstone of modern healthcare, aiding in diagnosis, treatment planning, and patient management. With the rapid advancement in technology, new and innovative solutions are continually being developed to enhance the capabilities of medical imaging. One such groundbreaking project is UltraMedical, an initiative by TsinghuaC3I, focusing on the application of ultrasound technology in medical imaging. This article will introduce UltraMedical, its features, benefits, and its potential impact on the medical field.
What is UltraMedical?
UltraMedical is an open-source project developed by TsinghuaC3I, aimed at leveraging ultrasound technology to improve medical imaging. The project is hosted on GitHub and provides a comprehensive suite of tools and resources for medical professionals, researchers, and developers to enhance their ultrasound imaging capabilities.
Key Features of UltraMedical
1. Advanced Image Processing
UltraMedical offers state-of-the-art image processing algorithms that enhance the clarity and detail of ultrasound images. These algorithms help in reducing noise, improving contrast, and highlighting important anatomical structures.
2. Real-Time Imaging
One of the standout features of UltraMedical is its ability to provide real-time imaging. This is particularly useful in clinical settings where immediate image feedback is crucial for accurate diagnosis and treatment.
3. User-Friendly Interface
The UltraMedical platform is designed with a user-friendly interface, making it accessible to medical professionals with varying levels of technical expertise. The intuitive design ensures that users can easily navigate through the different features and tools.
4. Customizable and Extensible
UltraMedical is highly customizable, allowing users to tailor the platform to their specific needs. Additionally, it is extensible, meaning developers can build and integrate new functionalities to further enhance the system’s capabilities.
5. Open Source and Community-Driven
As an open-source project, UltraMedical encourages collaboration and contributions from the global medical and developer community. This collaborative approach ensures continuous improvement and innovation.
Benefits of UltraMedical
1. Enhanced Diagnostic Accuracy
By providing clearer and more detailed images, UltraMedical aids in improving diagnostic accuracy. This is particularly beneficial in detecting and diagnosing conditions that might be challenging to identify with conventional ultrasound technology.
2. Improved Patient Outcomes
Accurate and timely diagnosis is critical for effective treatment. With UltraMedical’s advanced imaging capabilities, healthcare providers can make more informed decisions, leading to better patient outcomes.
3. Cost-Effective Solution
UltraMedical offers a cost-effective alternative to expensive imaging technologies. Its open-source nature reduces the overall cost of implementation, making advanced medical imaging more accessible.
4. Facilitates Research and Development
Researchers and developers can leverage UltraMedical to explore new applications and improvements in ultrasound technology. The platform’s extensibility supports innovation and experimentation.
Frequently Asked Questions (FAQs)
1. What is UltraMedical?
UltraMedical is an open-source project by TsinghuaC3I, focusing on enhancing medical imaging through advanced ultrasound technology.
2. Who can use UltraMedical?
UltraMedical is designed for medical professionals, researchers, and developers interested in ultrasound imaging. Its user-friendly interface makes it accessible to users with varying levels of technical expertise.
3. How can I access UltraMedical?
You can access UltraMedical through its GitHub repository. The repository includes all the necessary tools, resources, and documentation to get started.
4. Is UltraMedical free to use?
Yes, UltraMedical is an open-source project, making it free to use, modify, and distribute.
5. Can I contribute to UltraMedical?
Absolutely! UltraMedical encourages contributions from the community. You can contribute by developing new features, improving existing ones, or providing feedback.
6. What are the system requirements for UltraMedical?
The system requirements for UltraMedical depend on the specific use case and the components being utilized. Detailed requirements are available in the project’s documentation on GitHub.
7. How does UltraMedical improve ultrasound imaging?
UltraMedical uses advanced image processing algorithms to enhance image clarity and detail, providing better diagnostic information compared to traditional ultrasound imaging.
8. Is technical support available for UltraMedical?
As an open-source project, technical support is primarily community-driven. Users can seek help and share knowledge through forums, GitHub issues, and other collaborative platforms.
Conclusion
UltraMedical is poised to revolutionize the field of medical imaging with its advanced ultrasound technology. By offering a cost-effective, customizable, and user-friendly solution, it has the potential to significantly improve diagnostic accuracy and patient outcomes. Whether you are a medical professional looking to enhance your imaging capabilities, a researcher exploring new frontiers, or a developer seeking to innovate, UltraMedical provides a robust platform to achieve your goals. Explore UltraMedical today and join the community driving the future of medical imaging.