|
| 1 | +--- |
| 2 | +hip: 1037 |
| 3 | +title: Protocol Buffer API Specification |
| 4 | +author: Joseph Sinclair <@jsync-swirlds> |
| 5 | +requested-by: Richard Bair <@rbair> |
| 6 | +type: Standards Track |
| 7 | +category: Service |
| 8 | +needs-tsc-approval: Yes |
| 9 | +status: Last Call |
| 10 | +created: 2024-08-09 |
| 11 | +discussions-to: https://github.com/hashgraph/hedera-improvement-proposal/discussions/1037 |
| 12 | +updated: 2024-01-08 |
| 13 | +--- |
| 14 | + |
| 15 | +## Abstract |
| 16 | +The current protocol buffer API definitions are technically correct, but lack |
| 17 | +usable or useful specification. We should add, in comments compatible with |
| 18 | +protoc-gen-doc, specification text that clearly states both requirement and |
| 19 | +expectation for every message, file, and field. This documentation should be |
| 20 | +automatically transformed to markdown documentation that is published to the |
| 21 | +Hedera API site. In the ideal case this specification will be sufficient for |
| 22 | +any qualified team to implement a completely independent consensus node that |
| 23 | +is fully interoperable with the current consensus node software. |
| 24 | + |
| 25 | +## Motivation |
| 26 | +The existing protocol buffer API documentation is brief, entirely descriptive, |
| 27 | +and often inaccurate. This produces two negative consequences. First, it is not |
| 28 | +possible to produce a compatible independent implementation of the Hedera |
| 29 | +consensus node software without extensive reference to the existing source |
| 30 | +code. Second, developers are often unclear how best to interact with the |
| 31 | +Hedera API, and independent SDK developers, in particular, may struggle to |
| 32 | +correctly implement these API calls. |
| 33 | + |
| 34 | +As part of expanding the Hedera ecosystem we MUST reduce the difficulty and |
| 35 | +hurdles for independent developers when building systems that interact with |
| 36 | +the Hedera network. Clear and accurate API specifications are one element |
| 37 | +required to accomplish this goal. |
| 38 | + |
| 39 | +## Rationale |
| 40 | +This HIP improves the developer experience and enhances decentralization by |
| 41 | +making it easier to design distributed applications or alternative |
| 42 | +implementations of core network systems. |
| 43 | + |
| 44 | +The use of a markdown format that is automatically generated from the |
| 45 | +functional protocol buffer definition files encourages more frequent updates |
| 46 | +that are more closely tied to the functional updates to the API documents, and |
| 47 | +reduces the inevitable "drift" between functional API and API specification. |
| 48 | + |
| 49 | +## User stories |
| 50 | +- As a dApp developer I want to have clear and accurate specification for all |
| 51 | + Hedera APIs so that I can confidently design my application to call those |
| 52 | + APIs. |
| 53 | +- As an independent implementor I want to have clear and accurate specification |
| 54 | + for all Hedera APIs so that I can design an independent, correct, and |
| 55 | + compatible implementation of Hedera network nodes, including a consensus node. |
| 56 | +- As an Hedera software contributor I want to have clear and accurate |
| 57 | + specification for all Hedera APIs so that I can confidently implement changes |
| 58 | + to Hedera-led software. |
| 59 | +- As a dApp developer that receives bytes or is expected to parse, produce, or |
| 60 | + process bytes representing a HAPI protobuf message I want to have a clear |
| 61 | + specification of the structure and implications of the corresponding proto |
| 62 | + messages to understand how to correctly parse them. |
| 63 | + |
| 64 | +## Specification |
| 65 | +The core of this HIP is |
| 66 | +[the guidelines](../assets/hip-1037/Specification-Format-Style-Guidelines.md) |
| 67 | +recommended for this, and all future Hedera protocol buffer specifications. |
| 68 | +This HIP, however, also implements sweeping changes to specification text for |
| 69 | +the existing API documents. These changes affect over 20,000 lines of text |
| 70 | +across almost 150 files in order to add full specification text that complies |
| 71 | +with the guidelines expressed in this improvement proposal. The full reference |
| 72 | +implementation, therefore, is only linked here for brevity and clarity.<br/> |
| 73 | +The implementation of this HIP is detailed in github as a |
| 74 | +pull request linked [later in this document](#reference-implementation). |
| 75 | + |
| 76 | +## Backwards Compatibility |
| 77 | +This HIP does not change any functional characteristic, and documents the |
| 78 | +state of the API as-is. This does not introduce any change in compatibility. |
| 79 | + |
| 80 | +## Security Implications |
| 81 | +This HIP updates and clarifies specification and expectation for the Hedera |
| 82 | +API. This does not materially impact the security of the system. |
| 83 | + |
| 84 | +## How to Teach This |
| 85 | +The Hedera API is a critical component of the design and interaction for the |
| 86 | +Hedera ecosystem. Improvements to the formality and completeness of the API |
| 87 | +specification help everyone involved to develop distributed applications |
| 88 | +more quickly and with greater confidence. |
| 89 | + |
| 90 | +## Reference Implementation |
| 91 | +We have a |
| 92 | +[pull request](https://github.com/hashgraph/hedera-protobufs/pull/388) |
| 93 | +that implements these guidelines across the full hedera-protobufs repository. |
| 94 | + |
| 95 | +## Rejected Ideas |
| 96 | +1. Use of a highly formal "language" for specification |
| 97 | + * This was deemed excessive. Protocol Buffers are already a highly formal |
| 98 | + language, and adding specification as markdown comments, using the |
| 99 | + light weight RFC/HIP language is sufficient while remaining approachable |
| 100 | + for the largest practical number of community members. |
| 101 | +1. Addition of specialized "tag" elements to specification comments. |
| 102 | + * This would have required building a complete custom documentation |
| 103 | + processor and would have been non-standard. Rather than do so we chose |
| 104 | + to adhere closely to existing standard approaches for protocol buffers. |
| 105 | + |
| 106 | + |
| 107 | +## Open Issues |
| 108 | +None. |
| 109 | + |
| 110 | +## References |
| 111 | +- Protocol Buffer Document Generator |
| 112 | + [protoc-gen-doc](https://github.com/pseudomuto/protoc-gen-doc/?tab=readme-ov-file#protoc-gen-doc). |
| 113 | + |
| 114 | +## Copyright/license |
| 115 | +This document is licensed under the Apache License, Version 2.0 -- |
| 116 | +see [LICENSE](../LICENSE) or (https://www.apache.org/licenses/LICENSE-2.0) |
| 117 | + |
0 commit comments