7.1 KiB
Defining the Problem & Current State of the Art
Evaluator 1: Superior out of Superior "The submission addresses the challenge of reliable and secure user authentication in denied or sophisticated threat environments, particularly on edge devices. Current state-of-the-art capabilities and regrets are clearly identified."
Evaluator 2: Superior out of Superior "The submission does an excellent job of describing the problem of authentication - especially is use cases involving edge4 devices under stressful conditions. The submission further does a very nice job describing the state-of-the art, i.e. passwords and biometric, and highlighting the challenges associated with these approaches."
Evaluator 3: Highly Satisfactory out of Superior "Summary: Current password approaches to system security presents vulnerabilities Strengths: - Traditional “passwords” are recognized as being prone to hacking and abuse. This is particularly the case where “strong” passwords are not enforced. - “Lazy” users frequently re-use passwords across many sites. - Password expiration creates a burden on users to update and memorize frequent changes to many protected sites, making. Areas to improve: - Many approaches to include random generation of “strong” passwords tied to personal user credentials, dual/multi-factor authentication, biometrics and tokens/CAC cards have helped mitigate traditional PW risks. - The significance of specific tactical (or strategic) system vulnerabilities to AI-orchestrated cyberattacks or other threats was only generally addressed.
Advancing the State of the Art
Evaluator 1: Satisfactory out of Superior "The submission presents a solution to advance the state-of-the-art of authentication by employing a pictograph-based system that is encrypted using an industry-standard stream cypher (ChaCha20). The presented system has several attractive features that simplify reliable authentication in the field, where there are challenges with standard biometric techniques. The white space chart clearly indicates the presented system’s capability benefits compared to legacy password approaches. The submission, however, does not present any technology development of the system beyond scaling up the number of icons, so there is no apparent DARPA role."
Evaluator 2: Marginal out of Superior "I struggled with this rating because I think the submission presents an innovative solution to the authentication problem, I just think DARPA may not be the right customer given where they are in the development cycle. It looks like the technology is largely developed - the submission does not do a good job of describing the technical challenges DARPA funding will help them overcome, beyond adapting it for tactical edge devices and integration."
Evaluator 3: Satisfactory out of Superior "Summary: The submission advocates for a new paradigm for user authentication that is “more secure, intuitive and scalable” than traditional passwords. Strengths:
- Innovative approach to site protection using AI
- generated icons making “passwords” easier to remember, even across many different applications and domains and preventing PW re-use.
- The visual UI is patented and customizable, with development customized based on operational feedback from fintech, defense and cybersecurity.
- The submission suggest alignment to ESIR’s topic area for advanced technologies for improved resilience, efficiency and effectiveness of strategic systems. Areas to improve:
- The submission suggests that biometrics are inferior to the presented technology but often provides highly effective authentication in tactical and other environments.
- The technology is indicated to be TRL 5, with the greatest risks to adoption being integration, training, and “evolving AI threats”. These could be addressed as DOTMLPF gaps verses a need additional transformative research.
- Research goals of the submission lack definition and are generally focused on field trials verses discovery.
Team Capability (Key Personnel Vision, Expertise, and Experience)
Evaluator 1: Superior out of Superior "A very capable project team includes university cyber center personnel, professors with expertise in mathematics and computer science, and members with military experience. The team has already developed and successfully demonstrated a prototype product."
Evaluator 2: Superior out of Superior "The presented team has excellent credentials covering all the relevant expertise required to make the solution a reality - cybersecurity, math, software, operational expertise, etc."
Evaluator 3: Highly Satisfactory out of Superior "Summary: The submitter is a small business affiliated with two not-for-profits focused on cybersecurity. Strengths: - The team has already developed the technology and has IP protection for it. - Partnership provide additional depth of expertise and infrastructure to support development. - The submitting team is indicated to have substantial expertise and experience in cybersecurity, intelligence operations, red teaming, mathematical frameworks, and software development. Areas to improve: - Additional expertise or partnerships with military end-users that would support trials of the technology would strengthen the submission. - Project vision is not indicated.
Defense and/or Commercial Market Use Case/Impact
Evaluator 1: Highly Satisfactory out of Superior "The simplified and robust authentication process would improve user experience and enhance security for military users in a denied or cyber-threat environment."
Evaluator 2: Highly Satisfactory out of Superior "I can imagine a wide variety of military and commercial use cases where this technology would be be very beneficial, especially in a contested environment with operators under stress. The main reason I did not rate as superior is because I am concerned there might be some military applications where it would be difficult to see the small pictograph icons."
Evaluator 3: Highly Satisfactory out of Superior "Summary: The submission indicates broad applicability for the solution, to include tactical edge authentication and any commercial application where traditional PW authentication is employed. Strengths: - A range of defense use cases are indicated including tactical edge authentication, focused on the TAK application. - Commercial applications are suggested to be vast, with healthcare and banking as two exemplars. - Application of the solution of have the impact of more secure systems that are easier for end-users to access. Areas to improve: - None noted.
Coaches Feedback (Mike Cooper)
- make the demo at the end bigger. push it forward to advancing the state of the art
- testomonials go a long way (go and ask soldiers what they'd use). maybe emphasize User Insight. Discuss operational gap with biometrics
- they don't know what we're going to do with the money. (should borrow from NSF)
- add a slide that says "advancing the state of the art?"
- we'll likely get 2 of the three evaluators
- make sure we are at a 4 but not above. say we're going to take their money and get us to a TRL 6
- Don't be afraid be technical