mirror of
https://github.com/karma-riuk/crab-webapp.git
synced 2025-07-05 06:08:13 +02:00
284 lines
12 KiB
HTML
284 lines
12 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
|
|
<head>
|
|
<meta charset="UTF-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
|
<link rel="icon" type="image/x-icon" href="/img/crab.png">
|
|
<title>Dataset Downloader & Answer Uploader</title>
|
|
<link rel="stylesheet" href="css/style.css">
|
|
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
|
|
<script src="https://cdn.socket.io/4.5.4/socket.io.min.js"></script>
|
|
<script defer src="js/index.js"></script>
|
|
<script defer src="js/sorttable.js"></script>
|
|
<script defer src="js/modal.js"></script>
|
|
</head>
|
|
|
|
<body>
|
|
<header class="site-header">
|
|
<h3 id="page-title">
|
|
<img src="/img/crab.png" alt="Crab" class="crab-icon">
|
|
Crab Webapp
|
|
</h3>
|
|
<button id="about-button" class="info-btn">About</button>
|
|
</header>
|
|
<main>
|
|
|
|
<fieldset>
|
|
<legend>
|
|
<strong>Download a Dataset</strong>
|
|
<button id="info-download-btn" class='info-btn'><i class="fa fa-info"></i></button>
|
|
</legend>
|
|
|
|
<label for="dataset-select">Dataset:</label>
|
|
<select id="dataset-select">
|
|
<option value="comment_generation">Comment Generation</option>
|
|
<option value="code_refinement">Code Refinement</option>
|
|
</select>
|
|
<label>
|
|
<input type="checkbox" id="with-context">
|
|
Include context
|
|
</label>
|
|
<br /><br />
|
|
<button id="download-dataset">Download</button>
|
|
</fieldset>
|
|
|
|
<fieldset>
|
|
<legend>
|
|
<strong>Upload Your Answers</strong>
|
|
<button id="info-upload-btn" class='info-btn'><i class="fa fa-info"></i></button>
|
|
</legend>
|
|
<label for="answer-type">Type:</label>
|
|
<select id="answer-type">
|
|
<option value="comment">Comment Generation</option>
|
|
<option value="refinement">Code Refinement</option>
|
|
</select>
|
|
<br /><br />
|
|
<input type="file" id="file-input" accept="application/json" />
|
|
<br /><br />
|
|
<div style="display: flex; align-items: center; gap: 0.5em">
|
|
<button id="upload-btn">Upload JSON</button>
|
|
<div id="upload-status" class="hidden" style="color: green;"> hello world </div>
|
|
</div>
|
|
</fieldset>
|
|
|
|
<fieldset>
|
|
<legend>
|
|
<strong>Get status of ongoing process</strong>
|
|
<button id="info-results-btn" class='info-btn'><i class="fa fa-info"></i></button>
|
|
</legend>
|
|
<div style="display: flex; align-items: center; gap: 0.5em">
|
|
<label for="uuid">Process id:</label>
|
|
<input type="text" id="uuid" required />
|
|
</div>
|
|
<br /><br />
|
|
<button id="request-status">Request status</button>
|
|
<span id="status-status" class="hidden" style="color: green;"> </span>
|
|
</fieldset>
|
|
|
|
|
|
<div id="progress-container" class="hidden">
|
|
<h3>Processing answers...</h3>
|
|
<div>
|
|
<progress id="progress-bar" value="0" max="100"></progress>
|
|
<span id="progress-text">0%</span>
|
|
</div>
|
|
</div>
|
|
|
|
<div id="comment" class="results-container hidden">
|
|
<h3>Results Comment Generation<button class="download-results">Download</button></h3>
|
|
<table class="sortable">
|
|
<thead>
|
|
<tr>
|
|
<th>id</th>
|
|
<th>Proposed comment</th>
|
|
<th>Correct file</th>
|
|
<th>Distance</th>
|
|
<th>Max bleu score</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div id="refinement" class="results-container hidden">
|
|
<h3>Results Code Refinement<button class="download-results">Download</button></h3>
|
|
<table class="sortable">
|
|
<thead>
|
|
<tr>
|
|
<th>id</th>
|
|
<th>Compiled</th>
|
|
<th>Tested</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div id="modal-overlay" class="modal-overlay hidden" tabindex=-1>
|
|
<div class="modal-container">
|
|
<button id="modal-close" class="modal-close" aria-label="Close">×</button>
|
|
<div id="modal-content"></div>
|
|
</div>
|
|
</div>
|
|
|
|
<template id="about">
|
|
<h2>About this project</h2>
|
|
<div>
|
|
<p>CRAB (Code Review Automation Benchmark) is a research-driven platform designed to evaluate deep
|
|
learning models for code review tasks. Developed as part of a master's thesis at the Università
|
|
della Svizzera italiana, CRAB provides a high-quality, curated benchmark dataset of Java code review
|
|
triplets: submitted code, reviewer comment, and revised code. Each instance is manually validated to
|
|
ensure that reviewer comments directly address code issues and that the revised code implements the
|
|
feedback accurately. </p>
|
|
|
|
<p>The platform supports two core tasks: generating human-like review comments and refining code based
|
|
on those comments. It also accounts for paraphrased feedback and alternative valid code revisions,
|
|
offering a more realistic and robust evaluation. CRAB addresses the shortcomings of existing
|
|
datasets by eliminating noise and ensuring functional correctness through testing. Researchers can
|
|
upload model predictions to receive standardized evaluations, making CRAB an essential tool for
|
|
advancing automated code review technologies.</p>
|
|
|
|
</div>
|
|
</template>
|
|
|
|
<template id="info-download">
|
|
<h2>Downloading the Dataset</h2>
|
|
<div>
|
|
<p>
|
|
When you download a dataset, you'll receive a ZIP archive containing a JSON file. The structure of
|
|
this file depends on the selected task.
|
|
</p>
|
|
|
|
<section class="json-schemas">
|
|
<details>
|
|
<summary><strong>Comment Generation</strong></summary>
|
|
<p>The JSON maps each ID to an object with:</p>
|
|
<ul>
|
|
<li><strong>files</strong>: a map of filenames to their content at the start of the pull
|
|
request.</li>
|
|
<li><strong>diffs</strong>: a map of filenames to the diff that was applied to each file
|
|
before
|
|
the comment was made.</li>
|
|
</ul>
|
|
<pre><code>{
|
|
"1234": {
|
|
"files": {
|
|
"src/Main.java": "public class Main { ... }"
|
|
},
|
|
"diffs": {
|
|
"src/Main.java": "@@ -1,3 +1,6 @@ ..."
|
|
}
|
|
}
|
|
}</code></pre>
|
|
</details>
|
|
|
|
<details>
|
|
<summary><strong>Code Refinement</strong></summary>
|
|
<p>The JSON structure is similar to comment generation, with one additional field:</p>
|
|
<ul>
|
|
<li><strong>files</strong>: the initial version of each file in the PR.</li>
|
|
<li><strong>diffs</strong>: the diff applied before the comment was made.</li>
|
|
<li><strong>comments</strong>: a list of comments, each with a body, the file it refers to,
|
|
and
|
|
the exact location of the comment.</li>
|
|
</ul>
|
|
<pre><code lang="json">{
|
|
"5678": {
|
|
"files": { ... },
|
|
"diffs": { ... },
|
|
"comments": [
|
|
{
|
|
"body": "Consider simplifying this logic.",
|
|
"file": "src/Util.java",
|
|
"location": {
|
|
"start_line": 42,
|
|
"end_line": 45
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}</code></pre>
|
|
</details>
|
|
</section>
|
|
|
|
<h3>With Context (Optional)</h3>
|
|
<p>
|
|
You can choose to download the dataset with full repository context — the state of the entire
|
|
codebase at the time the PR was created. This may help your model better understand the broader
|
|
project structure and dependencies outside of the changed files.
|
|
</p>
|
|
</div>
|
|
</template>
|
|
<template id="info-upload">
|
|
<h2>Uploading Results</h2>
|
|
<section>
|
|
<p>
|
|
After downloading a dataset and generating your predictions for either task, you can upload your
|
|
results here to start the evaluation process.
|
|
</p>
|
|
<p>
|
|
Your uploaded JSON file must follow one of the schemas described below, depending on the selected
|
|
task. Once uploaded, the system will begin evaluating your submission. A progress bar will appear to
|
|
show how far along the evaluation is.
|
|
</p>
|
|
<p>
|
|
For <strong>Code Refinement</strong>, an id will also be displayed. This id allows you to safely
|
|
close the browser tab and later check the evaluation progress by pasting it into the <em>"Get
|
|
status of ongoing process"</em> section. More information is available in that section.
|
|
</p>
|
|
<section class="json-schemas">
|
|
<details>
|
|
<summary>
|
|
<strong>Comment Generation</strong>
|
|
</summary>
|
|
<p>
|
|
Submit a JSON object where each key is a string ID and each value is the generated comment.
|
|
</p>
|
|
<pre><code>
|
|
{
|
|
"1234": "This method lacks null checks.",
|
|
"5678": "Consider renaming this variable for clarity."
|
|
}
|
|
</code></pre>
|
|
|
|
</details>
|
|
<details>
|
|
<summary><strong>Code Refinement</strong></summary>
|
|
<p>
|
|
Submit a JSON object where each key is a string ID, and the value is another object that
|
|
maps
|
|
file
|
|
paths (relative to the top-level directory) to the new file content.
|
|
</p>
|
|
<pre><code>
|
|
{
|
|
"1234": {
|
|
"src/Main.java": "public class Main { /* updated code */ }"
|
|
},
|
|
"5678": {
|
|
"utils/Helper.java": "public class Helper { /* improved logic */ }"
|
|
}
|
|
}
|
|
</code></pre>
|
|
|
|
<p>
|
|
Make sure your file strictly follows the expected format to avoid upload errors.
|
|
</p>
|
|
</details>
|
|
</section>
|
|
</section>
|
|
</template>
|
|
<template id="info-results">
|
|
<h2>Getting information of ongoing process</h2>
|
|
<div>
|
|
yes
|
|
</div>
|
|
</template>
|
|
</main>
|
|
</body>
|
|
|
|
</html>
|