Skip to content
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
7776701
Add base image for Java benchmarks on OpenWhisk
mahlashrifi Sep 8, 2024
13aa9ec
Add Dockerfile for running Java benchmarks on OpenWhisk
mahlashrifi Sep 8, 2024
b026577
Update base image of java on OpenWhisk
mahlashrifi Sep 8, 2024
d478dc1
Add Java-based handler for OpenWhisk
mahlashrifi Sep 10, 2024
d36481c
Add example config file for running java benchmarks on OpenWhisk
mahlashrifi Sep 19, 2024
a7055e1
Add JAVA enum to list of languages
mahlashrifi Sep 19, 2024
cdd1763
Add config of 601.hello-world (A simple java benchmark)
mahlashrifi Sep 19, 2024
e9b72b2
Init maven structure of 601.hello-world and add some codes for runnin…
mahlashrifi Sep 19, 2024
8d2e015
Sync hello-world maven paroject with the wrapper of openwhisk
mahlashrifi Sep 19, 2024
8dcddef
Example config file for running 601.hello-world on openwhisk
mahlashrifi Sep 23, 2024
b6c8bb7
Correct Structure of maven project in 601.hello-world benchmark
mahlashrifi Sep 23, 2024
3032978
Expand add_code functions for maven java rojects
mahlashrifi Sep 23, 2024
32235f5
Exclude Java main wrapper from Docker directory created in runtimes.
mahlashrifi Sep 23, 2024
7f8eb2f
Fix a big
mahlashrifi Sep 25, 2024
67bb80c
Add required changes from PR222 to enable benchmarking of java codes.
mahlashrifi Mar 15, 2025
19c572d
Fix bug: Java simple benchmark (601.hello_world) now works correctly
mahlashrifi Mar 18, 2025
ab2bc2d
Use language enum instead of hardcoded 'java'
mahlashrifi Mar 18, 2025
37c37ae
Remove unused parts from the Java benchmarks wrapper for OpenWhisk.
mahlashrifi Mar 18, 2025
601903f
Change the directory where the file is created in container for detec…
mahlashrifi Mar 19, 2025
2c59db1
Add Java wrapper for AWS Lambda benchmarks
mahlashrifi May 4, 2025
726e07a
Add the nosql_func argument to the signature of generate_input
mahlashrifi May 10, 2025
4f88a8d
Update hello_world benchmark input/output) for platform-independence
mahlashrifi May 10, 2025
139cf29
Update system.json for Java on AWS
mahlashrifi May 10, 2025
b1a0a0f
Platform-related dependencies for Java benchmarks are now added dynam…
mahlashrifi May 10, 2025
5625faf
Add dockefile and installer for java on AWS
mahlashrifi May 14, 2025
f784af3
Adopt the output dir of Java wrappers for compatibility with Maven's …
mahlashrifi May 14, 2025
e7fad85
Update hashing to adapt to Java benchmark directory structure
mahlashrifi May 14, 2025
f1796c1
Java HelloWorld benchmark now runs on AWS (requires enhancement)
mahlashrifi May 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -188,3 +188,6 @@ cache
# IntelliJ IDEA files
.idea
*.iml

# Visual Studio Code files
.vscode/
6 changes: 6 additions & 0 deletions benchmarks/600.java/601.hello-world/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"timeout": 60,
"memory": 256,
"languages": ["java"]
}

5 changes: 5 additions & 0 deletions benchmarks/600.java/601.hello-world/input.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
def buckets_count():
return (0, 0)

def generate_input(data_dir, size, benchmarks_bucket, input_paths, output_paths, upload_func):
return { }
45 changes: 45 additions & 0 deletions benchmarks/600.java/601.hello-world/java/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>faas</groupId>
<artifactId>benchmark</artifactId>
<version>1</version>

<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
<!-- https://mvnrepository.com/artifact/com.google.code.gson/gson -->
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.11.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package faas;
import com.google.gson.JsonObject;

public class App {
public JsonObject handler(JsonObject args) {

JsonObject jsonResult = new JsonObject();
jsonResult.addProperty("Hello", "World");
return jsonResult;
}
}
75 changes: 75 additions & 0 deletions benchmarks/wrappers/aws/java/Handler.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.core.type.TypeReference;

import faas.App;

import java.io.File;
import java.io.IOException;
import java.time.Instant;
import java.util.HashMap;
import java.util.Map;

public class Handler implements RequestHandler<Map<String, Object>, String> {
private static final ObjectMapper mapper = new ObjectMapper();

@Override
public String handleRequest(Map<String, Object> event, Context context) {

Map<String, Object> inputData = event;

// Extract input if trigger is API Gateway (body is a string)
if (event.containsKey("body") && event.get("body") instanceof String)
try {
inputData = mapper.readValue((String) event.get("body"),new TypeReference<Map<String, Object>>() {});
} catch (IOException e) {
throw new RuntimeException("Failed to parse JSON body", e);
}

App function = new App();

Instant begin = Instant.now();
long start_nano = System.nanoTime();

Map<String, Object> functionOutput = function.handler(inputData);

long end_nano = System.nanoTime();
Instant end = Instant.now();


long computeTime = end_nano - start_nano;
// Detect cold start
boolean isCold = false;
String fileName = "/tmp/cold_run";

File file = new File(fileName);
if (!file.exists()) {
isCold = true;
try {
file.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
}

// Convert to Unix timestamp in seconds.microseconds
String formattedBegin = String.format("%d.%06d", begin.getEpochSecond(), begin.getNano() / 1000); // Convert nanoseconds to microseconds
String formattedEnd = String.format("%d.%06d", end.getEpochSecond(), end.getNano() / 1000);


Map<String, Object> result = new HashMap<>();
result.put("begin", formattedBegin);
result.put("end", formattedEnd);
result.put("request_id", context.getAwsRequestId());
result.put("compute_time", computeTime);
result.put("is_cold", isCold);
result.put("result", functionOutput);
try {
return mapper.writeValueAsString(result);
} catch (IOException e) {
throw new RuntimeException("Failed to serialize result of benchmark to JSON in Wrapper", e);
}

Comment on lines +61 to +73
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider adding function execution error handling.

The code doesn't handle potential exceptions thrown by the function. Consider wrapping the function invocation in a try-catch block to capture and include any function execution errors in the result.

        App function = new App();

        Instant begin = Instant.now();
        long start_nano = System.nanoTime();

-       Map<String, Object> functionOutput = function.handler(inputData);
+       Map<String, Object> functionOutput;
+       String errorMessage = null;
+       try {
+           functionOutput = function.handler(inputData);
+       } catch (Exception e) {
+           functionOutput = new HashMap<>();
+           errorMessage = e.getMessage();
+           // Continue execution to capture timing data even if function fails
+       }

        long end_nano = System.nanoTime();
        Instant end = Instant.now();


        long computeTime = end_nano - start_nano;
        // Rest of the code remains the same...

        Map<String, Object> result = new HashMap<>();
        // Add other fields as before...
        result.put("result", functionOutput);
+       if (errorMessage != null) {
+           result.put("error", errorMessage);
+       }

}
}
55 changes: 55 additions & 0 deletions benchmarks/wrappers/openwhisk/java/Main.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import faas.App;
import com.google.gson.JsonObject;
import java.time.Instant;
import java.time.Duration;
import java.io.File;
import java.io.IOException;


public class Main {
public static JsonObject main(JsonObject args) {

App function = new App();

long start_nano = System.nanoTime();

Instant begin = Instant.now();
JsonObject result = function.handler(args);
Instant end = Instant.now();

long end_nano = System.nanoTime();

// long computeTime = Duration.between(begin, end).toNanos() / 1000; // Convert nanoseconds to microseconds

long computeTime = end_nano - start_nano;
boolean isCold = false;
String fileName = "/tmp/cold_run";

File file = new File(fileName);
if (!file.exists()) {
isCold = true;
try {
file.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
}

// Convert to Unix timestamp in seconds.microseconds
String formattedBegin = String.format("%d.%06d", begin.getEpochSecond(), begin.getNano() / 1000); // Convert nanoseconds to microseconds
String formattedEnd = String.format("%d.%06d", end.getEpochSecond(), end.getNano() / 1000);

String requestId = System.getenv("__OW_ACTIVATION_ID");

JsonObject jsonResult = new JsonObject();
jsonResult.addProperty("begin", formattedBegin);
jsonResult.addProperty("end", formattedEnd);
jsonResult.addProperty("request_id", requestId);
jsonResult.addProperty("compute_time", computeTime);
jsonResult.addProperty("is_cold", isCold);
jsonResult.addProperty("result", result.toString());
return jsonResult;
}

}

Empty file.
69 changes: 69 additions & 0 deletions config/example2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
{
"experiments": {
"deployment": "openwhisk",
"update_code": false,
"update_storage": false,
"download_results": false,
"runtime": {
"language": "java",
"version": "8"
},
"type": "invocation-overhead",
"perf-cost": {
"benchmark": "601.hello-world",
"experiments": ["cold", "warm", "burst", "sequential"],
"input-size": "test",
"repetitions": 50,
"concurrent-invocations": 50,
"memory-sizes": [128, 256]
},
"network-ping-pong": {
"invocations": 50,
"repetitions": 1000,
"threads": 1
},
"invocation-overhead": {
"repetitions": 5,
"N": 20,
"type": "payload",
"payload_begin": 1024,
"payload_end": 6251000,
"payload_points": 20,
"code_begin": 1048576,
"code_end": 261619712,
"code_points": 20
},
"eviction-model": {
"invocations": 1,
"function_copy_idx": 0,
"repetitions": 5,
"sleep": 1
}
},
"deployment": {
"openwhisk": {
"shutdownStorage": false,
"removeCluster": false,
"wskBypassSecurity": "true",
"wskExec": "wsk",
"experimentalManifest": false,
"docker_registry": {
"registry": "",
"username": "",
"password": ""
},
"storage": {
"address": "",
"mapped_port": 9011,
"access_key": "",
"secret_key": "",
"instance_id": "",
"output_buckets": [],
"input_buckets": [],
"type": "minio"
}

}
}
}
Comment on lines +43 to +68
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Review security settings and sensitive information handling

The deployment configuration for OpenWhisk is comprehensive. However, there are some security considerations to address:

  1. The wskBypassSecurity is set to "true". This could pose a security risk if used in a production environment.
  2. Sensitive information fields (Docker registry credentials, storage access keys) are left empty. Ensure you have a secure method to populate these fields in different environments.

Consider the following improvements:

  1. Set wskBypassSecurity to false unless absolutely necessary for testing purposes.
  2. Implement a secure method to inject sensitive information (e.g., environment variables, secrets management system) rather than hardcoding them in the configuration file.
  3. Add a comment in the file to indicate how sensitive information should be handled.

Example:

"wskBypassSecurity": false,
// ... other fields ...
"docker_registry": {
  "registry": "", // Set via DOCKER_REGISTRY env var
  "username": "", // Set via DOCKER_USERNAME env var
  "password": ""  // Set via DOCKER_PASSWORD env var
},


18 changes: 18 additions & 0 deletions config/systems.json
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,24 @@
"minio": "7.0.16"
}
}
},
"java": {
"base_images": {
"8": "openwhisk/java8action"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we support newer Java versions, e.g., 11 or 17?

},
"images": [
"function"
],
"username": "docker_user",
"deployment": {
"files": [
"Main.java",
"Storage.java"
],
"packages": {
"minio": "8.5.9"
}
Comment on lines +355 to +357
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Validate the Minio version for Java.

The configuration specifies Minio version 8.5.9 for Java OpenWhisk functions. Ensure this version is compatible with Java 8 and verify if there are any known security vulnerabilities in this version.


🌐 Web query:

Is Minio Java client version 8.5.9 compatible with Java 8, and are there any security vulnerabilities?

💡 Result:

Yes, MinIO Java Client version 8.5.9 is compatible with Java 8. The MinIO Java SDK requires Java 1.8 or above. (github.com)

Regarding security vulnerabilities, version 8.5.9 has known issues due to dependencies on outdated libraries:

  • Guava Library Vulnerabilities: The Guava version used in this code base has known vulnerabilities. (github.com)

  • Simple-XML Library Vulnerability: The Simple-XML library version 2.7.1, used in this client, has an active CVE (CVE-2017-1000190). (github.com)

Additionally, the Maven Repository lists several vulnerabilities associated with version 8.5.9, including CVE-2024-34447, CVE-2024-30172, and CVE-2024-30171. (mvnrepository.com)

To mitigate these issues, it's recommended to upgrade to the latest version of the MinIO Java Client, which addresses these vulnerabilities. The latest version as of now is 8.5.17. (min.io)

Citations:


Upgrade MinIO Java Client to Address Known Vulnerabilities

The config/systems.json entry on lines 355–357 specifies MinIO Java Client 8.5.9, which is compatible with Java 8 but has the following known security issues:

To mitigate these risks, upgrade to the latest MinIO Java Client (currently 8.5.17):

--- config/systems.json
+++ config/systems.json
@@ lines 355-357
-          "packages": {
-            "minio": "8.5.9"
-          }
+          "packages": {
+            "minio": "8.5.17"
+          }

References:

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"packages": {
"minio": "8.5.9"
}
"packages": {
"minio": "8.5.17"
}

}
}
},
"architecture": ["x64"],
Expand Down
8 changes: 8 additions & 0 deletions dockerfiles/openwhisk/java/Dockerfile.function
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
ARG BASE_IMAGE
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dockerfile is very good :-) But we should also be able to deploy the code as zip package; particularly for cloud platforms that do not support containers (Azure, GCP). Can you add that or should I take a look into that?

FROM $BASE_IMAGE
COPY . /function/

# RUN apt-get update && apt-get install -y maven

# # Check if pom.xml exists before running Maven
# RUN if [ -f ./pom.xml ]; then mvn clean install; else echo "pom.xml not found, aborting build." && exit 1; fi
2 changes: 1 addition & 1 deletion sebs.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def simplified_common_params(func):
@click.option(
"--language",
default=None,
type=click.Choice(["python", "nodejs"]),
type=click.Choice(["python", "nodejs", "java"]),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Potential Inconsistencies: Missing "java" in Other Language Options

Several files still list only "python" and "nodejs" as language options. Please update the following files to include "java":

  • tools/build_docker_images.py
  • scripts/run_experiments.py
  • tests/aws/create_function.py
  • dockerfiles/local/runner.py
🔗 Analysis chain

LGTM: Java language support added

The change adds "java" as an option for the --language parameter, which is consistent with the PR objective of adding Java support to the SeBS framework.

To ensure this change is properly integrated, let's verify if there are any other parts of the code that might need updates to support Java:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for other occurrences of language-related code
rg --type python 'language.*python.*nodejs' sebs.py
rg --type python 'language.*python.*nodejs' .

Length of output: 163


Script:

#!/bin/bash
# Search for other occurrences of language-related code with correct file type
rg --type py 'language.*python.*nodejs' sebs.py
rg --type py 'language.*python.*nodejs' .

Length of output: 1219

help="Benchmark language",
)
@click.option("--language-version", default=None, type=str, help="Benchmark language version")
Expand Down
Loading