Create components

Once you have defined the task README and designed an API for the input/outputs of components and their file formats, the next step is to start adding components to your task. In this guide, we will walk you through the process of adding components to your new task, using the “create component” guide as a reference. The “create component” guide includes detailed instructions on how to:

• Add a method
• Add a control method
• Add a metric
• Run tests
• Create a pull request

Iterating over each of these steps will help you create the necessary components for your new task. Creating a small Bash script that runs an example component of each component type will also be useful for testing and ensuring that everything works as expected.

Step 1: Add a method

Methods are the algorithms or approaches you want to benchmark in your task. To add a new method, follow the guide on how to create a method. Ensure that your method is compatible with the input/output format defined in your API and test it thoroughly.

Step 2: Add a control method

Control method methods serve as reference points for evaluating the performance of other methods in your task. They can be positive or negative controls, depending on their intended purpose. To add a control method method, follow the guide on how to create a control method. Make sure your control method method adheres to the input/output format defined in your API and test it for correctness.

Step 3: Add a metric

Metrics are used to evaluate the performance of methods on your task. To add a new metric, follow the guide on how to create a metric. Ensure that your metric is compatible with the input/output format defined in your API and test it to confirm its accuracy.

Step 4: Run tests

Testing is a crucial part of the development process, ensuring that all components of your task function correctly and produce the expected results. To run tests on your components, follow the guide on how to run tests. Regularly running tests as you develop your task will help you identify and fix any issues early on, leading to a more robust and reliable benchmarking task.

Next steps

Once you have iterated through these steps and have a working set of components for your task, you can proceed to integrate them into the OpenProblems framework and submit your task for review by creating a pull request.