Bench 2023

The 15th BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing (Bench 2023)
In conjunction with Federated Intelligent Computing and Chip Conference (FICC 2023)
https://www.benchcouncil.org/bench2023/index.html

Full Papers: July 31, 2023, at 11:59 PM AoE
Notification: September 30, 2023, at 11:59 PM AoE
Final Papers Due: October 31, 2023, at 11:59 PM AoE
Conference Date: December 3-5, 2023
Venue: Sanya, China.

Please note that citizens from up to 59 nations can visit Sanya without a Visa from the Chinese Government. Sanya is a beautiful seaside city, well known as Hawaii in China.

Submission website: https://bench2023.hotcrp.com/

Introduction

Evolving from nine BPOE/SDBA workshops in conjunction with ASPLOS, VLDB, and ICS, Bench is an international multidisciplinary conference on benchmarks, standards, data sets, evaluation, and optimization. Bench 2023 is the fifteenth edition. The Bench conference encompasses a wide range of topics in benchmarks, datasets, metrics, indexes, measurement, evaluation, optimization, supporting methods and tools, and industry best practices in computer science, AI, medicine, finance, education, management, etc. Bench’s multidisciplinary and interdisciplinary emphasis provides an ideal environment for developers and researchers from different areas and communities to discuss practical and theoretical work.

Bench 2023 invites manuscripts describing original work in the above areas and topics (Call for Papers). All accepted papers will be presented at the Bench 2023 conference and published by Springer LNCS (Pending, Indexed by EI). At least one of the authors of the TBench articles published last year is requested to present their work at the Bench conference.

Regularly, the Bench conference will present the BenchCouncil Achievement Award ($3000), the BenchCouncil Rising Star Award ($1000), the BenchCouncil Best Paper Award ($1000), and the BenchCouncil Distinguished Doctoral Dissertation Awards in Computer Architecture ($1000) and in other areas ($1000). This year, the BenchCouncil Distinguished Doctoral Dissertation Award includes two tracks: computer architecture and other areas. Among the submissions of each track, four candidates will be selected as finalists. They will be invited to give a 30-minute presentation at the Bench 2023 Conference and contribute research articles to BenchCouncil Transactions on Benchmarks, Standards and Evaluation. Finally, for each track, one among the four will receive the award for each track, which carries a $1,000 honorarium.

With generous support from BenchCouncil, Bench 2023 will offer travel grants for students to defray a portion of their travel cost. The size and number of these grants will vary depending on funding availability, the number of student applicants, and their respective priority. Grant awards will be made before the early registration deadline; expenses will be reimbursed after the conference; grant recipients will be asked to submit original receipts to verify their expenditures as well as a 1-page summary of their involvement during the conference. While we encourage all in need of a travel grant to apply, the selection process will give higher priority to students who would otherwise not be able to attend the conference. We strongly encourage applications from students that belong to under-represented groups.

Organization

General Co-Chairs
Rakesh Agrawal, Data Insights Laboratories, San Jose, CA, USA
Aoying Zhou, East China Normal University

Program Co-Chairs
Weining Qian, East China Normal University
Sascha Hunold, TU Wien, Austria

Program Vice-Chairs
Biwei Xie, Institute of Computing Technology, CAS
Kai Shu, Illinois Institute of Technology

Web Chair
Jiahui Dai, BenchCouncil

Technical Program Committee(continuously updated):
Bin Ren, William & Mary
Guangli Li, Institute of Computing Technology, Chinese Academy of Sciences
Gwangsun Kim, POSTECH
Khaled Ibrahim, Lawrence Berkeley National Laboratory
Mario Marino, Leeds Beckett University
Miaoqing Huang, University of Arkansas
Murali Emani, Argonne National Laboratory
Vladimir Getov, University of Westminster
Woongki Baek, UNIST
Xiaoyi Lu, University of California, Merced
Zhen Jia, Amazon
Steven Farrell, Lawrence Berkeley National Laboratory

Award Committees

2023 BenchCouncil Achievement Award Committee:
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Since 2020)
Prof. David J. Lilja, University of Minnesota, Minneapolis (Since 2021)
Prof. Jack J. Dongarra, University of Tennessee (Since 2022)
John L. Henning, Oracle (Since 2023)

2023 BenchCouncil Rising Star Award Committees:
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Torsten Hoefler, ETH Zürich (Since 2021)
Prof. Vijay Janapa Reddi, Harvard University (Since 2022)
Dr. Peter Mattson, Google, USA (Since 2022)
Dr. Wanling Gao , ICT, Chinese Academy of Sciences (pending)
Dr. Douwe Kiela, Stanford University (Since 2023)

BenchCouncil Distinguished Doctoral Dissertation Award Committee in Other Areas:
Prof. Jack Dongarra, University of Tennessee
Dr. Xiaoyi Lu, The University of California, Merced
Dr. Jeyan Thiyagalingam, STFC-RAL
Dr. Lei Wang, ICT, Chinese Academy of Sciences
Dr. Spyros Blanas, The Ohio State University

BenchCouncil Distinguished Doctoral Dissertation Award Committee in Computer Architecture:
Prof. Resit Sendag, University of Rhode Island, USA
Dr. Peter Mattson, Google
Dr. Vijay Janapa Reddi, Harvard University
Dr. Wanling Gao, Chinese Academy of Sciences

Bench Steering Committees

Prof. Dr. Jack Dongarra, University of Tennessee
Prof. Dr. Geoffrey Fox, Indiana University
Prof. Dr. D. K. Panda, The Ohio State University
Prof. Dr. Felix, Wolf, TU Darmstadt.
JProf. Dr. Xiaoyi Lu, University of California, Merced
Prof. Dr. Resit Sendag, University of Rhode Island, USA
Dr. Wanling Gao, ICT, Chinese Academy of Sciences & UCAS
Prof. Dr. Jianfeng Zhan, BenchCouncil

Call for papers

The Bench conference encompasses a wide range of topics in benchmarks, datasets, metrics, indexes, measurement, evaluation, optimization, supporting methods and tools, and other best practices in computer science, medicine, finance, education, management, etc. Bench’s multidisciplinary and interdisciplinary emphasis provides an ideal environment for developers and researchers from different areas and communities to discuss practical and theoretical work. The topics of interest include, but are not limited to the following:

— Benchmark science and engineering across multi-disciplines: The formulation of problems or challenges in emerging and future computing; The benchmarks, datasets, and indexes in multidisciplinary applications, e.g., medical, finance, education, management, psychology, etc; Benchmark-based quantitative approaches to tackle multidisciplinary and interdisciplinary challenges; Industry best practices.

— Benchmark and standard specifications, implementations, and validations: Big Data, Artificial intelligence (AI), High performance computing (HPC), Machine learning, Big scientific data, Datacenter, Cloud, Warehouse-scale computing, Mobile robotics, Edge and fog computing, Internet of Things (IoT), Blockchain, Data management and storage, Financial, Education, Medical or other application domains.

— Dataset: Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements; Analyses or meta-analyses of existing data and original articles on systems, technologies, and techniques that advance data sharing and reuse to support reproducible research; Evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description; Tools that can generate large-scale data while preserving their original characteristics.

— Workload characterization, quantitative measurement, design, and evaluation studies: Computer and communication networks, protocols and algorithms; Wireless, mobile, ad-hoc and sensor networks, IoT applications; Computer architectures, hardware accelerators, multi-core processors, memory systems and storage networks; HPC systems; Operating systems, file systems and databases; Virtualization, data centers, distributed and cloud computing, fog and edge computing; Mobile and personal computing systems; Energy-efficient computing systems; Real-time and fault-tolerant systems; Security and privacy of computing and networked systems; Software systems and services, and enterprise applications; Social networks, multimedia systems, web services; Cyber-physical systems, including the smart grid.

— Methodologies, metrics, abstractions, algorithms, and tools: Analytical modeling techniques and model validation; Workload characterization and benchmarking; Performance, scalability, power and reliability analysis; Sustainability analysis and power management; System measurement, performance monitoring and forecasting; Anomaly detection, problem diagnosis and troubleshooting; Capacity planning, resource allocation, run time management and scheduling; Experimental design, statistical analysis, and simulation.

— Measurement and evaluation: Measurement standards; Evaluation methodologies and metrics; Testbed methodologies and systems; Instrumentation, sampling, tracing and profiling of large-scale, real-world applications and systems; Collection and analysis of measurement data that yield new insights; Measurement-based modeling (e.g., workloads, scaling behavior, assessment of performance bottlenecks); Methods and tools to monitor and visualize measurement and evaluation data; Systems and algorithms that build on measurement-based findings; Advances in data collection, analysis and storage (e.g., anonymization, querying, sharing); Reappraisal of previous empirical measurements and measurement-based conclusions; Descriptions of challenges and future directions that the measurement and evaluation community should pursue.

Paper Submission

Papers must be submitted in PDF. For a full paper, the page limit is 15 pages in the LNCS format, not including references. For a short paper, the page limit is 8 pages in the LNCS format, not including references. The review process follows a strict double-blind policy per the established Bench conference norms. The submissions will be judged based on the merit of the ideas rather than the length. After the conference, the proceedings will be published by Springer LNCS (Pending, Indexed by EI). Please note that the LNCS format is the final one for publishing.

At least one author must pre-register for the symposium, and at least one author must attend the symposium to present the paper. Papers for which no author is pre-registered will be removed from the proceedings.

Formatting Instructions

Please make sure your submission satisfies ALL of the following requirements:
· All authors and affiliation information must be anonymized.
· Paper must be submitted in printable PDF format.
· Please number the pages of your submission.
· The submission must be formatted for black-and-white printers. Please make sure your figures are readable when printed in black and white.
· The submission must describe unpublished work that is not currently under review of any other conference or journal venues.

Submission site: https://bench2023.hotcrp.com/
LNCS latex template: https://www.benchcouncil.org/file/llncs2e.zip

Awards

BenchCouncil Achievement Award ($3,000)
– This award recognizes a senior member who has made long-term contributions to benchmarking, measuring, and optimizing. The winner is eligible for the status of a BenchCouncil Fellow.

BenchCouncil Rising Star Award ($1,000)
– This award recognizes a junior member who demonstrates outstanding potential for research and practice in benchmarking, measuring, and optimizing.

BenchCouncil Best Paper Award ($1,000)
– This award recognizes a paper presented at the Bench conferences, which demonstrates potential impact on research and practice in benchmarking, measuring, and optimizing.

BenchCouncil Distinguished Doctoral Dissertation Award ($2000)
– This award recognizes and encourages superior research and writing by doctoral candidates in the broad field of benchmarks, data, standards, evaluations, and optimizations community. This year, the award includes two tracks, including the BenchCouncil Distinguished Doctoral Dissertation Award in Computer Architecture ($1000) and BenchCouncil Distinguished Doctoral Dissertation Award in other areas ($1000).