Table 2

F1 Scores for Random Forests, by Specialty and Type of Training Data

Specialty	F1 Score, Mean (95% CI)
Specialty	Combined	Prescription Only	Procedure Only
Allergy/immunology	0.912	0.860^a	0.903^a
Allergy/immunology	(0.906-0.917)	(0.855-0.865)	(0.901-0.905)
Anesthesiology	0.951	0.624^a	0.963^a
Anesthesiology	(0.950-0.952)	(0.615-0.632)	(0.962-0.964)
Cardiology	0.938	0.917^a	0.929^a
Cardiology	(0.935-0.940)	(0.913-0.919)	(0.927-0.932)
Dermatology	0.966	0.940^a	0.964^b
Dermatology	(0.964-0.968)	(0.937-0.943)	(0.963-0.965)
Emergency medicine	0.897	0.716^a	0.914^a
Emergency medicine	(0.894-0.899)	(0.708-0.723)	(0.911-0.916)
Endocrinology	0.865	0.869^a	0.623^a
Endocrinology	(0.858-0.871)	(0.863-0.875)	(0.615-0.630)
Gastroenterology	0.923	0.901^a	0.900^a
Gastroenterology	(0.922-0.924)	(0.897-0.905)	(0.897-0.903)
Hematology-oncology	0.874	0.872	0.677^a
Hematology-oncology	(0.872-0.876)	(0.869-0.876)	(0.673-0.680)
Infectious disease	0.745	0.758^a	0.474^a
Infectious disease	(0.740-0.750)	(0.754-0.763)	(0.462-0.486)
Nephrology	0.885	0.866^a	0.882^a
Nephrology	(0.882-0.887)	(0.864-0.868)	(0.879-0.884)
Neurology	0.885	0.895^a	0.732^a
Neurology	(0.883-0.887)	(0.892-0.897)	(0.725-0.739)
Neurosurgery	0.650	0.377^a	0.631^a
Neurosurgery	(0.645-0.656)	(0.364-0.389)	(0.626-0.635)
Obstetrics and gynecology	0.920	0.928^a	0.868^a
Obstetrics and gynecology	(0.917-0.923)	(0.927-0.929)	(0.865-0.870)
Ophthalmology	0.982	0.975^a	0.987^a
Ophthalmology	(0.982-0.982)	(0.974-0.976)	(0.987-0.987)
Orthopedic surgery	0.884	0.760^a	0.901^a
Orthopedic surgery	(0.880-0.888)	(0.756-0.765)	(0.900-0.903)
Otolaryngology	0.932	0.874^a	0.951^a
Otolaryngology	(0.927-0.937)	(0.868-0.880)	(0.949-0.952)
Pathology	0.987	0.005^a	0.990^a
Pathology	(0.986-0.987)	(0-0.012)	(0.990-0.990)
Physical medicine and rehabilitation	0.586	0.380^a	0.492^a
Physical medicine and rehabilitation	(0.583-0.589)	(0.374-0.387)	(0.483-0.500)
Plastic surgery	0.533	0.314^a	0.383^a
Plastic surgery	(0.527-0.539)	(0.287-0.341)	(0.378-0.388)
Primary care	0.920	0.911^a	0.878^a
Primary care	(0.917-0.922)	(0.910-0.912)	(0.876-0.880)
Psychiatry	0.930	0.938^a	0.740^a
Psychiatry	(0.929-0.932)	(0.936-0.940)	(0.734-0.745)
Pulmonology	0.836	0.843^a	0.818^a
Pulmonology	(0.834-0.837)	(0.839-0.848)	(0.814-0.822)
Radiation oncology	0.939	0.691^a	0.976^a
Radiation oncology	(0.933-0.945)	(0.680-0.702)	(0.975-0.977)
Radiology	0.979	0.275^a	0.984^a
Radiology	(0.977-0.980)	(0.272-0.279)	(0.983-0.985)
Rheumatology	0.916	0.916	0.726^a
Rheumatology	(0.913-0.919)	(0.913-0.918)	(0.719-0.733)
Surgery	0.774	0.624^a	0.735^a
Surgery	(0.767-0.781)	(0.614-0.634)	(0.731-0.740)
Urology	0.962	0.950^a	0.962
Urology	(0.958-0.965)	(0.947-0.953)	(0.961-0.963)
Macro F1	0.876	0.745^a	0.821^a
Macro F1	(0.874-0.878)	(0.741-0.748)	(0.820-0.823)

Note: Prescribing data are from 2014-2016 Centers for Medicare and Medicaid Services (CMS) Medicare Fee-For-Service Provider Utilization and Payment Data: Part D Prescriber Public Use Files.19 Procedure data from 2014-2016 CMS Medicare Fee-For-Service Provider Utilization and Payment Data: Physician and Other Supplier Public Use Files.²⁰
↵a P <.01 (paired t test vs combined).
↵b P <.05 (paired t test vs combined).