ISSTA2015-AE

We have submitted the artifact of μScalpel to ISSTA 2015 Artifact Evaluation track. Our submission was accepted. Here we show our artifact as submitted to ISSTA2015-AE, considering the reviewers' feedback.

Artifact

This artifact contains the tool μScalpel, which was used for autotransplantation. Also we provide our regression, augmented regression, and acceptance test suites, where possible. In the other cases these test suites were executed manually (for GUI programs or the Webserver), or the original regression test suite was not executing at all the organ.

For enhancing the reproducibility of our results, we include in this artifact all the required scripts, and tools, for obtaining:

• all the transplant in our empirical study and in the case study.
• the required time for doing each and every transplant, as well as the total time required for all our experiments (Table 2, columns under Time).
• the results of regression, augmented regression, and acceptance test suite (Table 2, columns PR, for Unanimously, Regression, Regression++, and Acceptance tests).
• the coverage results for the entire postoperative host, and just for the organs (Table 2, columns All and O, for Unanimously, Regression, Regression++, and Acceptance tests). )

Note: this scripts contains just one run of each experiment. In the paper the results are averaged on 20 runs. By using the random seed parameter of μScalpel, and than run the script for 20 times, the results from the paper may be approximated.

Instructions

For allowing the execution of our experiments in different environments, we have provided two ways for running μScalpel:

1. Binaries for Ubuntu 14.04:
   • Use the script located at: AllScripts/runAll.sh for running all of our experiments (the empirical study and the case study). After this, the results for Table 2 in the paper, are available at:
     • The file AllScripts/autotransplantationTimeInformation contains the timing information for all the transplants.
     • The results of the test suites and coverage information is available in all the individual experiments' folders.

       For example: the folder AllScripts/IDCT-DONOR-SCRIPTS/IDCT-CFLOW/TestSuites contains the results of the test suites for IDCT donor → CFLOW host transplant. The file ResultsCoverage/index.html contains the coverage information; the file orginalRegressionTestSuite.out contains the test suite available with the original host program; the file ResultsAcceptanceTestSuite.out contains the acceptance test suite; the file ResultsAugmentedRegression.out contains the manually augmented regression test suite, aimed at obtaining a higher coverage (on the postoperative host in general, and aimed specially at the organ's coverage).

   • The time required for running all the experiments, and obtaining all our results (although just for 1 run rather than 20 runs, as we report in the paper) is pretty big. It may be over 30 hours. Because of this, we have also provided scripts for running alone each of the experiments. For running only one experiment you have to call the script ./run.sh from the experiment's folder.

     For example: for running the IDCT donor → CFLOW host transplant you have to call the following script: AllScripts/IDCT-DONOR-SCRIPTS/IDCT-CFLOW.

   • Before running the scripts we check the environment and provide details about missing requirements for running μScalpel. This script is called just by the one that runs the entire experiments. Before running individual experiments, it is highly recommended that you run the script AllScripts/check.sh for identifying any missing hardware or software requirements. Without meeting μScalpel's, you might obtain different results, or μScalpel might crash.
2. Dockerfile to build a container with all the experiments:
   • For allowing the experiments to be run in different environments than Ubuntu 14.04, and for avoiding the installation of all the required tools, we provide a Dockerfile to build a Docker container. This container allows the running of the experiments in isolation, and guards the users of our artifact against its possible side effects.
   • We provided 2 scripts for a very easy set-up of the container. First, run DockerContainer/build.sh . This script builds the container and installs all the tools required by our experiments, starting from the provided Dockerfile. Second, for launching an interactive terminal, run: DockerContainer/run.sh . Inside the container, the experiments are at the path: /home/AllScripts. That folder's content as well as the process for running the experiments is identical with 1).
3. Docker container image:
   • For avoiding the overhead of building the Docker container from the Dockerfile, and also for ensuring that the dependencies of μScalpel are installed with the right version, as suggested by one of the reviewers, we provide an exported docker container image.
   • For starting up the exported docker container, run from the root of the artifact's folder: cat dockerContainer.tar | sudo docker import - development/muscalpel:latest. Further, proceed exactly as in the case of 2).

Obtaining results on different subjects

We have provided a template for running μScalpel. The folder muScalpelUsage contains this template. For using it, you have to add the code of the donor program in the folder: muScalpelUsage/CustomTransplant/Experiment/Transplant/Donor. The code of the host system should be in the folder: muScalpelUsage/CustomTransplant/Experiment/Transplant/Host . For running the transplant you need to do the following annotations:

• Add the annotation: __ADDGRAFTHERE__JUSTHERE at any program point in the host system. This is the implantation point in the host system.
• Chose a entry point for the organ, and provide it as a parameter to μScalpel.
• Generate the organ test suite, which is the only test suite required by μScalpel. An example of how an organ test suite might look like is at: muScalpelUsage/CustomTransplant/Experiment/Transplant/TransplantCode/main.c . It must be in that location, and the file must be called main.c. The check library should be used, as in the example file. In the test the user can check whatever he / she want. The user might use the GRAFT_INTERFACE function for the test cases, which is the wrapper of the organ. Its parameters are all the variable available at the insertion point in the host. After an initial run, these parameters might be checked in the file: muScalpelUsage/CustomTransplant/Experiment/Transplant/TransplantCode/InterfaceHeader.h.

  Note: all the files used by μScalpel and provided by the template must be overwritten: the files in muScalpelUsage/CustomTransplant/Experiment/Transplant/Donor, the files in muScalpelUsage/CustomTransplant/Experiment/Transplant/Host, the file muScalpelUsage/CustomTransplant/Experiment/Transplant/TransplantCode/main.c, the file muScalpelUsage/CustomTransplant/Experiment/Transplant/CFLAGS (which contains additional flags that are required by the compilation of the Donor if it the case).

• Use our helper script for running μScalpel. It takes the following parameters (in this order):
  1. The file that contains the implantation point in the host. For the provided example, this parameter is: Transplant/Host/main.c. All the paths must start from Transplant/ .
  2. The file that contains the entry point of the organ. For the provided example, this parameter is: Transplant/Donor/main.c.
  3. The entry point of the organ. For the provided example, this parameter is: bar.

  Note: the command for running the existing example is: ./run.sh Transplant/Host/main.c Transplant/Donor/main.c bar. This command must run from muScalpelUsage/CustomTransplant

  .

Observations

• We report here all the test suites used in the paper. However, the only required one for μScalpel to run is the organ's test suite (i.e. the one available at AllScripts/IDCT-DONOR-SCRIPTS/IDCT-CFLOW/Experiment/Cflow-IDCT_Transplant/TransplantCode/main.c, for the case of IDCT donor → CFLOW host transplant). The rest of the test suites are used just for the validation of the organ, and are exercised after the organ is extracted, and implanted into the host, at the implantation point. Although generating the organ test suite might be difficult, this test suite, together with the implantation point in the host, and organ's entry in the donor, are the only inputs required by μScalpel. If we compare these inputs to the manual work required for transplanting / implementing a complex organ (as is the case of h.264 encoder), we think that μScalpel might saves softer developers a lot of time, and manual effort, at the cost of computational time.
• In the case of CFLOW donor → CFLOW host transplant we manually stripped the entire functionality of generating call graphs from the CFLOW host. We have done so, for being able to do the identity transplant: we check if μScalpel is able to transplant back, exactly the same functionality, from a host program where it was completely removed.