Documentation and presentation of the code used for the Master's thesis titled "Expert-Guided Causal Discovery: Estimating a Causal Cohort Graph from IDEFICS/I.Family Study".

The repository is split into two parts:

1. Simulations, which contains the code to reproduce the simulation results, plots, and tables.
2. Application, which contains the code to apply EITL to the IDEFICS/I.Family study.

---

1. Simulations

This code was originally run on the Cluster in BIPS, and jobs are split according to its capacity. If it is run on a different system, some settings may need adjusting.

Files

• run_simulation.tmpl: Bash template for submitting jobs to the cluster. Used in simulation_main.R and simulation_shortcut.R. Also can be used to schedule the simulations by changing the begin parameter.
• simulation_main.R: Runs simulations for the original version of EITL.
• simulation_shortcut.R: Runs simulations for the shortcut version of EITL.
• simulation_plots.R: Creates the plots from Section 3.3.2 and Appendix C.
• test_cycles.R: Analysis of the specific simulation settings of cycles created. Also checks whether these cycles disappear for the "wrong addition" expert when the threshold is lowered to 0.01. (Which is easier than rerunning the simulations with a lower threshold.)

Before running simulation_main.R or simulation_shortcut.R, make sure to define the variable base_dir, and make sure this directory contains the following folders:

• load
• EITL
• simulation_results

No other changes should be needed. Results are saved into simulation_results (this repository also includes my simulation results).

---

2. Application

Code for applying EITL to the IDEFICS/I.Family data. This includes:

• data management of expert knowledge
• the EITL algorithm
• scripts for estimating the graphs
• scripts for the creation of the tables and figures used in the Application and Discussion sections

2.1 10_DataManagement

Handles the data in 20_Data.

• Knowledge_matrices.R creates the matrices shown in Figures 4.2, 4.5, and 4.6. These visualize:
  • the temporal and logical constraints from Foraita et al. (2024)
  • the edge directions provided by the two expert approaches (human and LLM)
• 01_PrepareListWithConstraints.R builds a half complete dataframe of domain knowledge consisting of forbidden edges and direction assignments where only one is allowed, using the constraints from Foraita et al. (2024).
• 02_CompleteListWithExpertKnowledge_Human.R completes the previous dataframe using the human expert's directions.
• 03_CompleteListWithExpertKnowledge_LLM.R completes the previous dataframe using the LLLM directions.

2.2 20_Data

Contains all background knowledge data used in the application.

• fg_Zülal.csv: Temporal and logical constraint matrix, provided by Ronja Foraita.
• answers.txt: Answers from the LLM prompts.
• pair_counts.rds: Total counts for each edge from the LLM prompts.
• Expert_knowledge/prompting.R: Automatic prompting to generate the LLM answers above. This requires a Gemini API key from the free tier.
• Expert_knowledge/ready_to_fill_directions.rds: Half completed dataframe, output of 01_PrepareListWithConstraints.R.
• Expert_knowledge/Domain_knowledge_human_MW.rds: Completed dataframe using the human expert, built from the manually filled 01_Human_directions_MW.xlsx.
• Expert_knowledge/Domain_knowledge_LLM.rds: Completed dataframe using the LLM expert, built from 02_LLM_directions.csv.

2.3 30_Analysis

Contains the code for estimating and analyzing the causal graphs.

00_EXPERTINTHELOOP

Folder containing the EITL algorithm.

• REITL.R: Wrapper function that handles the different algorithm versions and logging.

• RANKEDEXPERTINTHELOOP.R: EITL implementation, also has a shortcut version in the shortcut folder modified for its alternating strategy.

• RANKEDEXPAND.R: Ranked version of the EXPAND function as described in the thesis, also has a shortcut version in the shortcut folder modified for its alternating strategy.

• PRUNE.R: PRUNE implementation.

• FIXCYCLES.R: FIXCYCLES implementation, following the heuristic suggested by the authors.

• micdtest/ containts the modified versions of the functions above to work with flexCItest. This was done for the original version only.

• helper_functions/ contains supporting functions used in EITL:

  • pillai_trace_test.R and pillai_trace_test_capped.R are the two implementations of the CI test Pillai's trace: the original, and the shortcut version with the limited amount of trees.
  • make_key.R helps with caching
  • log.info.R creates log messages
  • bnlearn_to_dagitty.R converts a graph from a bnlearn object to a dagitty object (I needed this because I use functions from both packages together)
  • ci_helper.R is a wrapper for the Pillai's trace CI test
  • ci_helper_micd.R is a wrapper for the flexCItest CI test

• logs/ containts the log files created while estimating the four DAGs.

01_OriginalEITL/ and 02_Shortcut/ each contain two scripts to estimate graphs using EITL, one for the human expert and one for the LLM expert.

03_Graphanalysis/:

• 01_Visualization.R creates the graph visualizations in Figures 4.7 and 4.8.
• 02_Analysis.R provides the scripts for the graph analysis discussed in the Application and Discussion sections.

90_micdtest/ script to estimate a graph using flexCItest as the statistical test with EITL (Mentioned in Section 5.1 and Appendix E).