From 3807f2d573e15d667fe85dbadfb20279df5c82cf Mon Sep 17 00:00:00 2001 From: ivanmilevtues Date: Sat, 31 May 2025 03:32:22 +0200 Subject: [PATCH 1/3] Add CodeBoarding documentation --- CodeBoarding/Client Interface.md | 52 +++++++++++++++++++++ CodeBoarding/Cluster Support.md | 53 +++++++++++++++++++++ CodeBoarding/Command Abstraction.md | 66 +++++++++++++++++++++++++++ CodeBoarding/Connection Management.md | 66 +++++++++++++++++++++++++++ CodeBoarding/Data Handling.md | 54 ++++++++++++++++++++++ CodeBoarding/PubSub Management.md | 43 +++++++++++++++++ CodeBoarding/on_boarding.md | 50 ++++++++++++++++++++ 7 files changed, 384 insertions(+) create mode 100644 CodeBoarding/Client Interface.md create mode 100644 CodeBoarding/Cluster Support.md create mode 100644 CodeBoarding/Command Abstraction.md create mode 100644 CodeBoarding/Connection Management.md create mode 100644 CodeBoarding/Data Handling.md create mode 100644 CodeBoarding/PubSub Management.md create mode 100644 CodeBoarding/on_boarding.md diff --git a/CodeBoarding/Client Interface.md b/CodeBoarding/Client Interface.md new file mode 100644 index 0000000000..b5f9bae16d --- /dev/null +++ b/CodeBoarding/Client Interface.md @@ -0,0 +1,52 @@ +```mermaid +graph LR + Redis_Client["Redis Client"] + Connection_Pool["Connection Pool"] + Connection["Connection"] + Command_Parser["Command Parser"] + PubSub["PubSub"] + Pipeline["Pipeline"] + Monitor["Monitor"] + Redis_Client -- "creates" --> Connection_Pool + Redis_Client -- "uses" --> Command_Parser + Redis_Client -- "creates" --> PubSub + Redis_Client -- "creates" --> Pipeline + Redis_Client -- "creates" --> Monitor + Connection_Pool -- "uses" --> Connection + Pipeline -- "uses" --> Redis_Client + PubSub -- "uses" --> Redis_Client + Monitor -- "uses" --> Redis_Client +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The Redis client library provides a comprehensive interface for interacting with Redis servers, supporting both synchronous and asynchronous operations. It encompasses connection management, command execution, and response handling, offering features like PubSub, Pipelines, and Monitoring. The core components work together to provide a robust and efficient way to interact with Redis. + +### Redis Client +The Redis Client serves as the primary interface for interacting with a Redis server. It manages connections, executes commands, and handles responses. It supports both synchronous and asynchronous operations, allowing users to interact with Redis in a non-blocking manner. The client can be configured to connect to a single Redis instance or a Redis cluster. +- **Related Classes/Methods**: `redis.client.Redis` (112:670), `redis.asyncio.client.Redis` (109:715) + +### Connection Pool +The Connection Pool manages a pool of reusable connections to the Redis server. This improves performance by reducing the overhead of establishing new connections for each request. The connection pool handles connection creation, recycling, and error handling, ensuring efficient use of resources. +- **Related Classes/Methods**: `redis.connection.ConnectionPool` (1309:1654), `redis.asyncio.connection.ConnectionPool` (1031:1253) + +### Connection +The Connection class represents a single connection to the Redis server. It handles the low-level details of socket communication, including sending commands and receiving responses. It provides methods for reading and writing data to the socket, as well as handling connection errors. +- **Related Classes/Methods**: `redis.connection.Connection` (730:801), `redis.asyncio.connection.Connection` (723:777) + +### Command Parser +The Command Parser is responsible for parsing the responses received from the Redis server. It converts the raw byte strings into Python data types, such as strings, integers, lists, and dictionaries. The parser handles different response formats and error conditions, ensuring that the data is correctly interpreted. +- **Related Classes/Methods**: `redis.client.Redis.parse_response` (646:667), `redis.asyncio.client.Redis.parse_response` (689:715) + +### PubSub +The PubSub class provides functionality for publishing messages to channels and subscribing to channels to receive messages. It enables real-time communication between clients using the publish-subscribe pattern. It supports pattern subscriptions and message filtering. +- **Related Classes/Methods**: `redis.client.PubSub` (743:1241), `redis.asyncio.client.PubSub` (803:1231) + +### Pipeline +The Pipeline class allows batching multiple commands into a single request, reducing network overhead and improving performance. It supports transactions, allowing a group of commands to be executed atomically. It also supports optimistic locking using WATCH and UNWATCH commands. +- **Related Classes/Methods**: `redis.client.Pipeline` (1283:1635), `redis.asyncio.client.Pipeline` (1251:1618) + +### Monitor +The Monitor class allows you to listen to all requests received by the Redis server in real time. It's useful for debugging and monitoring Redis activity. It provides a stream of commands and their arguments as they are processed by the server. +- **Related Classes/Methods**: `redis.client.Monitor` (676:740), `redis.asyncio.client.Monitor` (730:800) \ No newline at end of file diff --git a/CodeBoarding/Cluster Support.md b/CodeBoarding/Cluster Support.md new file mode 100644 index 0000000000..c58179e45e --- /dev/null +++ b/CodeBoarding/Cluster Support.md @@ -0,0 +1,53 @@ +```mermaid +graph LR + RedisCluster["RedisCluster"] + NodesManager["NodesManager"] + ClusterPubSub["ClusterPubSub"] + ClusterPipeline["ClusterPipeline"] + PipelineStrategy["PipelineStrategy"] + TransactionStrategy["TransactionStrategy"] + ClusterMultiKeyCommands["ClusterMultiKeyCommands"] + RedisCluster -- "manages" --> NodesManager + RedisCluster -- "uses" --> ClusterPubSub + RedisCluster -- "uses" --> ClusterPipeline + RedisCluster -- "determines node for" --> ClusterMultiKeyCommands + NodesManager -- "provides node information to" --> RedisCluster + ClusterPubSub -- "executes commands" --> RedisCluster + ClusterPipeline -- "executes commands" --> RedisCluster + PipelineStrategy -- "implements" --> AbstractStrategy + TransactionStrategy -- "implements" --> AbstractStrategy + ClusterMultiKeyCommands -- "uses" --> RedisCluster +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The Cluster Support component in redis-py provides the necessary tools for interacting with Redis clusters. It abstracts the complexities of cluster management, such as node discovery, slot assignment, and command routing, offering a unified interface for both synchronous and asynchronous operations. The core of this component lies in the `RedisCluster` class, which acts as the primary client. It leverages the `NodesManager` to maintain an updated view of the cluster topology and uses strategies like `PipelineStrategy` and `TransactionStrategy` to optimize command execution. The component also includes specialized classes for pub/sub (`ClusterPubSub`) and multi-key commands (`ClusterMultiKeyCommands`), ensuring comprehensive cluster functionality. + +### RedisCluster +The RedisCluster class serves as the primary client interface for interacting with a Redis cluster. It handles connection management, command execution, and slot assignment to nodes in the cluster. It uses the NodesManager to maintain an up-to-date view of the cluster topology and routes commands to the appropriate nodes based on the key's slot. It supports both synchronous and asynchronous operations. +- **Related Classes/Methods**: `redis.cluster.RedisCluster` (456:1360), `redis.asyncio.cluster.RedisCluster` (99:989) + +### NodesManager +The NodesManager class is responsible for managing the cluster's node topology. It maintains a list of all nodes in the cluster, their roles (primary or replica), and the slot ranges they serve. It dynamically updates the topology when nodes are added or removed, ensuring that the RedisCluster always has an accurate view of the cluster's structure. +- **Related Classes/Methods**: `redis.cluster.NodesManager` (1443:1863), `redis.asyncio.cluster.NodesManager` (1211:1518) + +### ClusterPubSub +The ClusterPubSub class provides a client interface for interacting with Redis cluster's pub/sub functionality. It handles subscribing to channels and publishing messages to channels across the cluster, ensuring that messages are correctly routed to the appropriate nodes. +- **Related Classes/Methods**: `redis.cluster.ClusterPubSub` (1866:2107) + +### ClusterPipeline +The ClusterPipeline class enables the execution of a batch of commands in a pipeline on a Redis cluster. It efficiently routes commands to the correct nodes and executes them in parallel, reducing network overhead and improving performance. It supports both synchronous and asynchronous operations. +- **Related Classes/Methods**: `redis.cluster.ClusterPipeline` (2110:2299), `redis.asyncio.cluster.ClusterPipeline` (1521:1680) + +### PipelineStrategy +The PipelineStrategy class implements the AbstractStrategy interface for executing commands in a Redis cluster pipeline. It handles routing commands to the correct nodes and executing them in parallel, optimizing performance for pipelined operations. +- **Related Classes/Methods**: `redis.cluster.PipelineStrategy` (2671:3018), `redis.asyncio.cluster.PipelineStrategy` (1872:2039) + +### TransactionStrategy +The TransactionStrategy class implements the AbstractStrategy interface for executing commands in a Redis cluster transaction. It ensures that all commands within the transaction are executed atomically, providing data consistency and reliability. +- **Related Classes/Methods**: `redis.cluster.TransactionStrategy` (3021:3352), `redis.asyncio.cluster.TransactionStrategy` (2042:2398) + +### ClusterMultiKeyCommands +The ClusterMultiKeyCommands class provides methods for executing multi-key commands on a Redis cluster. It handles partitioning keys by slot and routing commands to the correct nodes, ensuring that multi-key operations are performed efficiently across the cluster. +- **Related Classes/Methods**: `redis.commands.cluster.ClusterMultiKeyCommands` (99:260), `redis.commands.cluster.AsyncClusterMultiKeyCommands` (263:339) \ No newline at end of file diff --git a/CodeBoarding/Command Abstraction.md b/CodeBoarding/Command Abstraction.md new file mode 100644 index 0000000000..26fbca8edf --- /dev/null +++ b/CodeBoarding/Command Abstraction.md @@ -0,0 +1,66 @@ +```mermaid +graph LR + Command_Interface["Command Interface"] + Basic_Key_Commands["Basic Key Commands"] + Hash_Commands["Hash Commands"] + List_Commands["List Commands"] + Set_Commands["Set Commands"] + Sorted_Set_Commands["Sorted Set Commands"] + Stream_Commands["Stream Commands"] + PubSub_Commands["PubSub Commands"] + Script_Commands["Script Commands"] + Basic_Key_Commands -- "Implements" --> Command_Interface + Hash_Commands -- "Implements" --> Command_Interface + List_Commands -- "Implements" --> Command_Interface + Set_Commands -- "Implements" --> Command_Interface + Sorted_Set_Commands -- "Implements" --> Command_Interface + Stream_Commands -- "Implements" --> Command_Interface + PubSub_Commands -- "Implements" --> Command_Interface + Script_Commands -- "Implements" --> Command_Interface + List_Commands -- "Uses" --> Basic_Key_Commands + Set_Commands -- "Uses" --> Basic_Key_Commands + Stream_Commands -- "Uses" --> Basic_Key_Commands + Sorted_Set_Commands -- "Uses" --> Basic_Key_Commands + Hash_Commands -- "Uses" --> Basic_Key_Commands +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The Command Abstraction component provides a unified interface for interacting with Redis, abstracting away the complexities of command encoding, decoding, and execution. It encompasses various command categories, including keys, hashes, lists, sets, sorted sets, streams, pubsub, scripts, geo, modules, and functions, offering both synchronous and asynchronous execution modes. This abstraction ensures a consistent API for developers, regardless of the underlying Redis connection type, and simplifies the process of interacting with the Redis server. + +### Command Interface +Defines the base interface for all Redis commands, providing a consistent way to execute commands and handle responses. It serves as a blueprint for concrete command implementations. +- **Related Classes/Methods**: `redis.commands.core.CommandsInterface` (20:100) + +### Basic Key Commands +Implements basic key-related commands such as GET, SET, EXISTS, and DELETE. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding key values. +- **Related Classes/Methods**: `redis.commands.core.BasicKeyCommands` (1557:2510) + +### Hash Commands +Implements commands for interacting with Redis hashes, including setting, getting, and deleting fields. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding hash field values. +- **Related Classes/Methods**: `redis.commands.core.HashCommands` (4921:5598) + +### List Commands +Implements commands for interacting with Redis lists, including pushing, popping, and trimming elements. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding list element values. +- **Related Classes/Methods**: `redis.commands.core.ListCommands` (2533:2947) + +### Set Commands +Implements commands for interacting with Redis sets, including adding, removing, and checking membership of elements. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding set element values. +- **Related Classes/Methods**: `redis.commands.core.SetCommands` (3287:3462) + +### Sorted Set Commands +Implements commands for interacting with Redis sorted sets, including adding, removing, and retrieving elements with scores. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding sorted set element values. +- **Related Classes/Methods**: `redis.commands.core.SortedSetCommands` (4077:4870) + +### Stream Commands +Implements commands for interacting with Redis streams, including adding messages, reading messages, and creating consumer groups. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding stream message values. +- **Related Classes/Methods**: `redis.commands.core.StreamCommands` (3468:4071) + +### PubSub Commands +Implements commands for interacting with Redis's Pub/Sub functionality, including publishing messages and subscribing to channels. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding pubsub messages. +- **Related Classes/Methods**: `redis.commands.core.PubSubCommands` (5720:5784) + +### Script Commands +Enables the execution of Lua scripts on the Redis server. It includes functionalities for evaluating, loading, and managing scripts, providing a way to extend Redis's capabilities with custom logic. +- **Related Classes/Methods**: `redis.commands.core.ScriptCommands` (5790:5928) \ No newline at end of file diff --git a/CodeBoarding/Connection Management.md b/CodeBoarding/Connection Management.md new file mode 100644 index 0000000000..abae81ce86 --- /dev/null +++ b/CodeBoarding/Connection Management.md @@ -0,0 +1,66 @@ +```mermaid +graph LR + AbstractConnection["AbstractConnection"] + Connection["Connection"] + SSLConnection["SSLConnection"] + UnixDomainSocketConnection["UnixDomainSocketConnection"] + ConnectionPool["ConnectionPool"] + BlockingConnectionPool["BlockingConnectionPool"] + AbstractConnection_asyncio_["AbstractConnection (asyncio)"] + Connection_asyncio_["Connection (asyncio)"] + ConnectionPool_asyncio_["ConnectionPool (asyncio)"] + SentinelConnectionPool["SentinelConnectionPool"] + AbstractConnection -- "is a base class for" --> Connection + AbstractConnection -- "is a base class for" --> SSLConnection + AbstractConnection -- "is a base class for" --> UnixDomainSocketConnection + ConnectionPool -- "manages" --> Connection + ConnectionPool -- "extends" --> BlockingConnectionPool + AbstractConnection_asyncio_ -- "is a base class for" --> Connection_asyncio_ + ConnectionPool_asyncio_ -- "manages" --> Connection_asyncio_ + SentinelConnectionPool -- "manages" --> Connection +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The Connection Management component in `redis` library is responsible for managing connections to Redis servers. It provides connection pooling, different connection types (TCP, SSL, Unix sockets), and connection management functionalities like authentication and health checks. The core of this component revolves around the `AbstractConnection` and its implementations, along with `ConnectionPool` which manages the connections. Sentinel-related classes handle connections in a Sentinel-managed Redis setup. + +### AbstractConnection +AbstractConnection defines the base connection interface and implements common connection logic. It handles connection establishment, health checks, command sending, and response reading. It serves as a parent class for concrete connection implementations, providing a foundation for different connection types. +- **Related Classes/Methods**: `redis.connection.AbstractConnection:__init__` (228:322), `redis.connection.AbstractConnection:__repr__` (324:326), `redis.connection.AbstractConnection:__del__` (332:336), `redis.connection.AbstractConnection:_construct_command_packer` (338:344), `redis.connection.AbstractConnection:connect` (377:379), `redis.connection.AbstractConnection:connect_check_health` (381:413), `redis.connection.AbstractConnection:_error_message` (423:424), `redis.connection.AbstractConnection:on_connect` (426:427), `redis.connection.AbstractConnection:on_connect_check_health` (429:538), `redis.connection.AbstractConnection:_send_ping` (560:564), `redis.connection.AbstractConnection:_ping_failed` (566:568), `redis.connection.AbstractConnection:check_health` (570:573), `redis.connection.AbstractConnection:send_packed_command` (575:604), `redis.connection.AbstractConnection:send_command` (606:611), `redis.connection.AbstractConnection:can_read` (613:625), `redis.connection.AbstractConnection:read_response` (627:669), `redis.connection.AbstractConnection:re_auth` (719:727) + +### Connection +Connection is a concrete implementation of AbstractConnection for standard TCP connections. It inherits connection management and command execution logic from AbstractConnection, providing a basic TCP socket connection to Redis. +- **Related Classes/Methods**: `redis.connection.Connection:__init__` (733:747) + +### SSLConnection +SSLConnection extends Connection to provide SSL/TLS encryption for secure communication with Redis. It handles SSL context creation and socket wrapping, ensuring secure data transmission. +- **Related Classes/Methods**: `redis.connection.SSLConnection:__init__` (1024:1096), `redis.connection.SSLConnection:_connect` (1098:1107), `redis.connection.SSLConnection:_wrap_socket_with_ssl` (1109:1186) + +### UnixDomainSocketConnection +UnixDomainSocketConnection extends Connection for communication over Unix domain sockets. It adapts the connection process to use socket files instead of TCP addresses, enabling local communication with Redis. +- **Related Classes/Methods**: `redis.connection.UnixDomainSocketConnection:__init__` (1192:1195) + +### ConnectionPool +ConnectionPool manages a pool of connections to a Redis server. It handles connection creation, retrieval, and release, optimizing connection reuse and reducing connection overhead. It also manages the encoder to use for the connections, improving performance and resource utilization. +- **Related Classes/Methods**: `redis.connection.ConnectionPool:from_url` (1324:1370), `redis.connection.ConnectionPool:__init__` (1372:1433), `redis.connection.ConnectionPool:_checkpid` (1467:1512), `redis.connection.ConnectionPool:get_connection` (1519:1551), `redis.connection.ConnectionPool:get_encoder` (1553:1560), `redis.connection.ConnectionPool:make_connection` (1562:1573), `redis.connection.ConnectionPool:release` (1575:1597), `redis.connection.ConnectionPool:disconnect` (1602:1620), `redis.connection.ConnectionPool:close` (1622:1624), `redis.connection.ConnectionPool:re_auth_callback` (1633:1646) + +### BlockingConnectionPool +BlockingConnectionPool extends ConnectionPool to provide blocking behavior when retrieving connections from the pool. It waits for a connection to become available if the pool is exhausted, ensuring that a connection is eventually obtained. +- **Related Classes/Methods**: `redis.connection.BlockingConnectionPool:__init__` (1691:1705), `redis.connection.BlockingConnectionPool:make_connection` (1731:1740), `redis.connection.BlockingConnectionPool:get_connection` (1747:1797), `redis.connection.BlockingConnectionPool:release` (1799:1818), `redis.connection.BlockingConnectionPool:disconnect` (1820:1824) + +### AbstractConnection (asyncio) +AbstractConnection (asyncio) defines the base connection interface and implements common connection logic for asynchronous connections. It handles connection establishment, health checks, command sending, and response reading in an asynchronous manner. +- **Related Classes/Methods**: `redis.asyncio.connection.AbstractConnection:__init__` (138:225), `redis.asyncio.connection.AbstractConnection:__del__` (227:241), `redis.asyncio.connection.AbstractConnection:__repr__` (251:253), `redis.asyncio.connection.AbstractConnection:connect` (294:296), `redis.asyncio.connection.AbstractConnection:connect_check_health` (298:338), `redis.asyncio.connection.AbstractConnection:_error_message` (348:349), `redis.asyncio.connection.AbstractConnection:on_connect` (354:356), `redis.asyncio.connection.AbstractConnection:on_connect_check_health` (358:464), `redis.asyncio.connection.AbstractConnection:disconnect` (466:487), `redis.asyncio.connection.AbstractConnection:_send_ping` (489:493), `redis.asyncio.connection.AbstractConnection:_ping_failed` (495:497), `redis.asyncio.connection.AbstractConnection:send_packed_command` (511:550), `redis.asyncio.connection.AbstractConnection:send_command` (552:556), `redis.asyncio.connection.AbstractConnection:can_read_destructive` (558:565), `redis.asyncio.connection.AbstractConnection:read_response` (567:623), `redis.asyncio.connection.AbstractConnection:pack_commands` (671:699), `redis.asyncio.connection.AbstractConnection:process_invalidation_messages` (705:707), `redis.asyncio.connection.AbstractConnection:re_auth` (712:720) + +### Connection (asyncio) +Connection (asyncio) is a concrete implementation of AbstractConnection for standard TCP connections in an asynchronous context. It inherits connection management and command execution logic from AbstractConnection, providing an asynchronous TCP socket connection to Redis. +- **Related Classes/Methods**: `redis.asyncio.connection.Connection:__init__` (726:741), `redis.asyncio.connection.Connection:_connect` (752:774) + +### ConnectionPool (asyncio) +ConnectionPool (asyncio) manages a pool of asynchronous connections to a Redis server. It handles connection creation, retrieval, and release, optimizing connection reuse and reducing connection overhead in asynchronous operations. +- **Related Classes/Methods**: `redis.asyncio.connection.ConnectionPool:from_url` (1046:1090), `redis.asyncio.connection.ConnectionPool:__init__` (1092:1112), `redis.asyncio.connection.ConnectionPool:get_connection` (1138:1148), `redis.asyncio.connection.ConnectionPool:get_available_connection` (1150:1159), `redis.asyncio.connection.ConnectionPool:make_connection` (1170:1172), `redis.asyncio.connection.ConnectionPool:ensure_connection` (1174:1188), `redis.asyncio.connection.ConnectionPool:release` (1190:1198), `redis.asyncio.connection.ConnectionPool:aclose` (1222:1224), `redis.asyncio.connection.ConnectionPool:re_auth_callback` (1232:1245) + +### SentinelConnectionPool +SentinelConnectionPool manages a pool of connections to Redis servers monitored by Sentinel. It handles connection discovery, failover, and role management based on Sentinel's information, ensuring high availability and fault tolerance. +- **Related Classes/Methods**: `redis.sentinel.SentinelConnectionPool:__init__` (145:166), `redis.sentinel.SentinelConnectionPool:reset` (175:177), `redis.sentinel.SentinelConnectionPool:owns_connection` (183:188), `redis.sentinel.SentinelConnectionPool:get_master_address` (190:191), `redis.sentinel.SentinelConnectionPool:rotate_slaves` (193:195) \ No newline at end of file diff --git a/CodeBoarding/Data Handling.md b/CodeBoarding/Data Handling.md new file mode 100644 index 0000000000..bb7c73e731 --- /dev/null +++ b/CodeBoarding/Data Handling.md @@ -0,0 +1,54 @@ +```mermaid +graph LR + Encoder["Encoder"] + CommandsParser["CommandsParser"] + BaseParser["BaseParser"] + _RESP2Parser["_RESP2Parser"] + _RESP3Parser["_RESP3Parser"] + _HiredisParser["_HiredisParser"] + SocketBuffer["SocketBuffer"] + Helpers["Helpers"] + Encoder -- "encodes data with" --> CommandsParser + SocketBuffer -- "reads from" --> _HiredisParser + SocketBuffer -- "reads from" --> BaseParser + _HiredisParser -- "parses responses with" --> Helpers + _RESP3Parser -- "parses responses with" --> Helpers + _RESP2Parser -- "parses responses with" --> Helpers +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The Data Handling component is responsible for managing the serialization of commands and parsing of responses between the client and the Redis server. It encompasses encoding outgoing commands into the Redis Serialization Protocol (RESP) format and decoding incoming responses back into Python data types. This involves handling different RESP versions (RESP2, RESP3) and utilizing helper functions for parsing specific data structures. The component ensures data integrity and compatibility during communication with the Redis server. + +### Encoder +The Encoder class handles the serialization of Python data into the Redis Serialization Protocol (RESP) format. It provides methods for encoding various data types, ensuring they are properly formatted for transmission to the Redis server. It interacts with the CommandsParser to encode commands before sending them to Redis. +- **Related Classes/Methods**: `redis._parsers.encoders.Encoder:encode` (14:35) + +### CommandsParser +The CommandsParser class is responsible for parsing commands and extracting keys from them. It is initialized with command flags and provides methods for retrieving keys based on the command being parsed. It interacts with the Encoder to prepare commands for encoding and transmission. +- **Related Classes/Methods**: `redis._parsers.commands.AbstractCommandsParser:_get_pubsub_keys` (11:38), `redis._parsers.commands.AbstractCommandsParser:parse_subcommand` (40:53), `redis._parsers.commands.CommandsParser:__init__` (65:67), `redis._parsers.commands.CommandsParser:get_keys` (82:145), `redis._parsers.asyncio.commands.AsyncCommandsParser:get_keys` (full file reference) + +### BaseParser +The BaseParser class serves as the base class for all parsers (RESP2, RESP3, Hiredis). It provides common functionality and defines the interface for parsing responses. It includes methods for handling errors and managing the connection. It interacts with the SocketBuffer to read data and with the Helpers module to parse specific data types. +- **Related Classes/Methods**: `redis._parsers.base.BaseParser:parse_error` (90:99), `redis._parsers.base._RESPBase:__del__` (117:121), `redis._parsers.base._RESPBase:on_connect` (123:129), `redis._parsers.base._RESPBase:can_read` (139:140), `redis._parsers.base._AsyncRESPBase:__init__` (218:223), `redis._parsers.base._AsyncRESPBase:on_connect` (229:236), `redis._parsers.base._AsyncRESPBase:can_read_destructive` (242:251), `redis._parsers.base._AsyncRESPBase:_read` (253:271), `redis._parsers.base._AsyncRESPBase:_readline` (273:289) + +### _RESP2Parser +The _RESP2Parser class is responsible for parsing RESP2 responses. It provides methods for reading and parsing responses according to the RESP2 protocol. It handles the parsing of data structures defined in RESP2. It inherits from BaseParser and uses the Helpers module for parsing specific data types. +- **Related Classes/Methods**: `redis._parsers.resp2._RESP2Parser:read_response` (12:22), `redis._parsers.resp2._RESP2Parser:_read_response` (24:68), `redis._parsers.resp2._AsyncRESP2Parser:read_response` (74:85), `redis._parsers.resp2._AsyncRESP2Parser:_read_response` (87:132) + +### _RESP3Parser +The _RESP3Parser class is responsible for parsing RESP3 responses. It provides methods for reading and parsing responses according to the RESP3 protocol. It handles the complexities of the RESP3 protocol, including different data types and structures. It inherits from BaseParser and uses the Helpers module for parsing specific data types. +- **Related Classes/Methods**: `redis._parsers.resp3._RESP3Parser:__init__` (18:21), `redis._parsers.resp3._RESP3Parser:read_response` (28:40), `redis._parsers.resp3._RESP3Parser:_read_response` (42:131), `redis._parsers.resp3._AsyncRESP3Parser:__init__` (135:138), `redis._parsers.resp3._AsyncRESP3Parser:read_response` (145:158), `redis._parsers.resp3._AsyncRESP3Parser:_read_response` (160:257) + +### _HiredisParser +The _HiredisParser class is a parser that uses the hiredis library for parsing RESP (Redis Serialization Protocol) responses. It provides methods for reading data from the socket and parsing the response. It leverages the speed and efficiency of the hiredis library for faster parsing. It inherits from BaseParser and uses the Helpers module for parsing specific data types. +- **Related Classes/Methods**: `redis._parsers.hiredis._HiredisParser:__init__` (44:51), `redis._parsers.hiredis._HiredisParser:__del__` (53:57), `redis._parsers.hiredis._HiredisParser:can_read` (92:100), `redis._parsers.hiredis._HiredisParser:read_from_socket` (102:130), `redis._parsers.hiredis._HiredisParser:read_response` (132:184), `redis._parsers.hiredis._AsyncHiredisParser:__init__` (192:199), `redis._parsers.hiredis._AsyncHiredisParser:can_read_destructive` (233:242), `redis._parsers.hiredis._AsyncHiredisParser:read_from_socket` (244:251), `redis._parsers.hiredis._AsyncHiredisParser:read_response` (253:295) + +### SocketBuffer +The SocketBuffer class manages reading data from a socket. It provides methods for reading lines, reading a specific number of bytes, checking if data is available, and purging the buffer. It acts as an intermediary between the raw socket and the parsers, providing a buffered interface for reading data. It is used by the parsers (RESP2, RESP3, Hiredis) to read data from the socket. +- **Related Classes/Methods**: `redis._parsers.socket.SocketBuffer:_read_from_socket` (47:92), `redis._parsers.socket.SocketBuffer:can_read` (94:97), `redis._parsers.socket.SocketBuffer:read` (99:108), `redis._parsers.socket.SocketBuffer:readline` (110:118), `redis._parsers.socket.SocketBuffer:purge` (132:149) + +### Helpers +The Helpers module contains a collection of helper functions for parsing various Redis responses. These functions are used to extract specific data from the responses and convert them into Python data types. They provide specialized parsing logic for different Redis commands and data structures. It is used by the parsers (RESP2, RESP3, Hiredis) to parse specific data types. +- **Related Classes/Methods**: `redis._parsers.helpers:parse_debug_object` (17:32), `redis._parsers.helpers:parse_info` (35:83), `redis._parsers.helpers:parse_memory_stats` (86:94), `redis._parsers.helpers:parse_sentinel_state` (124:137), `redis._parsers.helpers:parse_sentinel_master` (140:141), `redis._parsers.helpers:parse_sentinel_state_resp3` (144:154), `redis._parsers.helpers:parse_sentinel_masters` (157:162), `redis._parsers.helpers:parse_sentinel_masters_resp3` (165:166), `redis._parsers.helpers:parse_sentinel_slaves_and_sentinels` (169:170), `redis._parsers.helpers:parse_sentinel_slaves_and_sentinels_resp3` (173:174), `redis._parsers.helpers:parse_stream_list` (238:247), `redis._parsers.helpers:pairs_to_dict_with_str_keys` (250:251), `redis._parsers.helpers:parse_xclaim` (258:261), `redis._parsers.helpers:parse_xautoclaim` (264:268), `redis._parsers.helpers:parse_xinfo_stream` (271:299), `redis._parsers.helpers:parse_xread` (302:305), `redis._parsers.helpers:parse_xread_resp3` (308:311), `redis._parsers.helpers:parse_xpending` (314:323), `redis._parsers.helpers:bool_ok` (337:338), `redis._parsers.helpers:parse_client_list` (349:354), `redis._parsers.helpers:parse_config_get` (357:359), `redis._parsers.helpers:parse_hscan` (367:374), `redis._parsers.helpers:parse_slowlog_get` (389:415), `redis._parsers.helpers:parse_stralgo` (418:444), `redis._parsers.helpers:parse_cluster_info` (447:449), `redis._parsers.helpers:_parse_node_line` (452:472), `redis._parsers.helpers:parse_cluster_nodes` (495:502), `redis._parsers.helpers:parse_command` (541:557), `redis._parsers.helpers:parse_command_resp3` (560:578), `redis._parsers.helpers:parse_client_kill` (585:588), `redis._parsers.helpers:parse_acl_getuser` (591:631), `redis._parsers.helpers:parse_acl_log` (634:649), `redis._parsers.helpers:parse_client_info` (652:680), `redis._parsers.helpers:parse_config_get` (357:359), `redis._parsers.helpers:parse_client_info` (652:680), `redis._parsers.helpers:parse_set_result` (683:694) \ No newline at end of file diff --git a/CodeBoarding/PubSub Management.md b/CodeBoarding/PubSub Management.md new file mode 100644 index 0000000000..321174a318 --- /dev/null +++ b/CodeBoarding/PubSub Management.md @@ -0,0 +1,43 @@ +```mermaid +graph LR + PubSub_redis_client_["PubSub (redis.client)"] + PubSub_redis_asyncio_client_["PubSub (redis.asyncio.client)"] + Redis_redis_client_["Redis (redis.client)"] + Redis_redis_asyncio_client_["Redis (redis.asyncio.client)"] + PubSubCommands["PubSubCommands"] + ClusterPubSub_redis_cluster_["ClusterPubSub (redis.cluster)"] + Redis_redis_client_ -- "creates" --> PubSub_redis_client_ + Redis_redis_asyncio_client_ -- "creates" --> PubSub_redis_asyncio_client_ + PubSub_redis_client_ -- "uses" --> PubSubCommands + PubSub_redis_asyncio_client_ -- "uses" --> PubSubCommands + ClusterPubSub_redis_cluster_ -- "creates" --> PubSub_redis_client_ +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The PubSub Management component in Redis provides publish/subscribe functionalities for real-time messaging. Clients can subscribe to channels and receive messages published to those channels. It supports both synchronous and asynchronous pub/sub operations, managing subscriptions and efficiently distributing messages to subscribers. The core components include the synchronous and asynchronous PubSub classes, the Redis client classes that provide access to PubSub instances, the ClusterPubSub class for cluster environments, and the PubSubCommands class that defines the core pubsub commands. + +### PubSub (redis.client) +The PubSub class in redis.client provides a synchronous interface for subscribing to channels and listening for messages. It manages connection details, command execution, message parsing, and thread management for asynchronous message handling within a thread. +- **Related Classes/Methods**: `redis.client.PubSub` (743:1241), `redis.client.PubSub.__init__` (756:791), `redis.client.PubSub.__exit__` (796:797), `redis.client.PubSub.__del__` (799:806), `redis.client.PubSub:close` (823:824), `redis.client.PubSub:on_connect` (826:849), `redis.client.PubSub:execute_command` (856:880), `redis.client.PubSub:_execute` (910:921), `redis.client.PubSub:parse_response` (923:948), `redis.client.PubSub:psubscribe` (983:1007), `redis.client.PubSub:punsubscribe` (1009:1020), `redis.client.PubSub:subscribe` (1022:1046), `redis.client.PubSub:unsubscribe` (1048:1059), `redis.client.PubSub:ssubscribe` (1061:1085), `redis.client.PubSub:sunsubscribe` (1087:1098), `redis.client.PubSub:listen` (1100:1105), `redis.client.PubSub:get_message` (1107:1134), `redis.client.PubSub:ping` (1138:1143), `redis.client.PubSub:handle_message` (1145:1217), `redis.client.PubSub:run_in_thread` (1219:1241), `redis.client.PubSub:clean_health_check_responses` (882:898) + +### PubSub (redis.asyncio.client) +The PubSub class in redis.asyncio.client provides an asynchronous interface for subscribing to channels and listening for messages. It manages connections, executes commands, parses responses, and handles asynchronous message processing using asyncio. +- **Related Classes/Methods**: `redis.asyncio.client.PubSub` (803:1231), `redis.asyncio.client.PubSub.__init__` (816:855), `redis.asyncio.client.PubSub.__aexit__` (860:861), `redis.asyncio.client.PubSub:close` (885:887), `redis.asyncio.client.PubSub:reset` (890:892), `redis.asyncio.client.PubSub:on_connect` (894:910), `redis.asyncio.client.PubSub:execute_command` (917:927), `redis.asyncio.client.PubSub:connect` (929:947), `redis.asyncio.client.PubSub:_execute` (956:967), `redis.asyncio.client.PubSub:parse_response` (969:995), `redis.asyncio.client.PubSub:psubscribe` (1023:1042), `redis.asyncio.client.PubSub:punsubscribe` (1044:1057), `redis.asyncio.client.PubSub:subscribe` (1059:1078), `redis.asyncio.client.PubSub:unsubscribe` (1080:1092), `redis.asyncio.client.PubSub:listen` (1094:1099), `redis.asyncio.client.PubSub:get_message` (1101:1114), `redis.asyncio.client.PubSub:ping` (1116:1121), `redis.asyncio.client.PubSub:handle_message` (1123:1187), `redis.asyncio.client.PubSub:run` (1189:1231) + +### Redis (redis.client) +The Redis class in redis.client provides the base synchronous Redis client. It exposes the `pubsub` method, which returns a PubSub instance associated with that client, allowing clients to subscribe to channels and receive messages. +- **Related Classes/Methods**: `redis.client.Redis:pubsub` (556:564) + +### Redis (redis.asyncio.client) +The Redis class in redis.asyncio.client provides the base asynchronous Redis client. It exposes the `pubsub` method, which returns a PubSub instance associated with that client, enabling asynchronous subscription to channels and message reception. +- **Related Classes/Methods**: `redis.asyncio.client.Redis:pubsub` (578:586) + +### PubSubCommands +The PubSubCommands class in redis.commands.core provides the core pubsub commands such as publish, spublish, pubsub_channels, pubsub_numpat, pubsub_numsub, and pubsub_shardnumsub. These commands are used to publish messages to channels, retrieve information about active channels, and manage subscriptions. +- **Related Classes/Methods**: `redis.commands.core.PubSubCommands:publish` (5726:5733), `redis.commands.core.PubSubCommands:spublish` (5735:5742), `redis.commands.core.PubSubCommands:pubsub_channels` (5744:5750), `redis.commands.core.PubSubCommands:pubsub_shardchannels` (5752:5758), `redis.commands.core.PubSubCommands:pubsub_numpat` (5760:5766), `redis.commands.core.PubSubCommands:pubsub_numsub` (5768:5775), `redis.commands.core.PubSubCommands:pubsub_shardnumsub` (5777:5784) + +### ClusterPubSub (redis.cluster) +The ClusterPubSub class in redis.cluster provides an interface for pubsub operations in a Redis Cluster environment. It handles sharded messages and manages connections to multiple nodes to ensure messages are correctly distributed across the cluster. +- **Related Classes/Methods**: `redis.cluster.RedisCluster:pubsub` (834:839), `redis.cluster.ClusterPubSub` (1866:2107), `redis.cluster.ClusterPubSub.__init__` (1875:1918), `redis.cluster.ClusterPubSub:set_pubsub_node` (1920:1951), `redis.cluster.ClusterPubSub:_raise_on_invalid_node` (1959:1967), `redis.cluster.ClusterPubSub:execute_command` (1969:2009), `redis.cluster.ClusterPubSub:get_sharded_message` (2033:2057), `redis.cluster.ClusterPubSub:ssubscribe` (2059:2077), `redis.cluster.ClusterPubSub:sunsubscribe` (2079:2091) \ No newline at end of file diff --git a/CodeBoarding/on_boarding.md b/CodeBoarding/on_boarding.md new file mode 100644 index 0000000000..dbc624c0d4 --- /dev/null +++ b/CodeBoarding/on_boarding.md @@ -0,0 +1,50 @@ +```mermaid +graph LR + Client_Interface["Client Interface"] + Command_Abstraction["Command Abstraction"] + Connection_Management["Connection Management"] + Cluster_Support["Cluster Support"] + PubSub_Management["PubSub Management"] + Data_Handling["Data Handling"] + Client_Interface -- "Uses" --> Connection_Management + Client_Interface -- "Uses" --> Command_Abstraction + Client_Interface -- "Uses" --> Data_Handling + Cluster_Support -- "Uses" --> Connection_Management + Command_Abstraction -- "Uses" --> Client_Interface + PubSub_Management -- "Uses" --> Client_Interface + click Client_Interface href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Client Interface.md" "Details" + click Command_Abstraction href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Command Abstraction.md" "Details" + click Connection_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Connection Management.md" "Details" + click Cluster_Support href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Cluster Support.md" "Details" + click PubSub_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/PubSub Management.md" "Details" + click Data_Handling href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Data Handling.md" "Details" +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) + +## Component Details + +The redis-py library provides a Python interface for interacting with Redis, a popular in-memory data structure store. The library abstracts the complexities of the Redis protocol, offering a user-friendly API for performing various operations, including data manipulation, pub/sub, and cluster management. It supports both synchronous and asynchronous operations, catering to a wide range of application requirements. The library is designed to be efficient and reliable, providing robust connection management and error handling. + +### Client Interface +The Client Interface serves as the primary entry point for interacting with Redis. It encapsulates connection management, command execution, and response handling. Supporting both synchronous and asynchronous operations, it can be configured to connect to a single Redis instance or a Redis cluster. This component handles the core functionality of sending commands to the Redis server and receiving responses, providing a high-level API for users. +- **Related Classes/Methods**: `redis.client.Redis` (112:670), `redis.asyncio.client.Redis` (109:715) + +### Command Abstraction +The Command Abstraction offers a high-level interface for executing Redis commands, encompassing implementations for various command categories like keys, hashes, lists, sets, and sorted sets. It supports both synchronous and asynchronous execution, providing a consistent API for interacting with Redis regardless of the underlying connection type. This component also manages command encoding and decoding, ensuring seamless communication with the Redis server. +- **Related Classes/Methods**: `redis.commands.core.BasicKeyCommands` (1557:2510), `redis.commands.core.HashCommands` (4921:5598), `redis.commands.core.ListCommands` (2533:2947), `redis.commands.core.SetCommands` (3287:3462), `redis.commands.core.SortedSetCommands` (4077:4870), `redis.commands.core.StreamCommands` (3468:4071), `redis.commands.core.PubSubCommands` (5720:5784), `redis.commands.core.AsyncBasicKeyCommands` (2513:2530) + +### Connection Management +The Connection Management component is responsible for establishing and maintaining connections to the Redis server. It provides connection pooling, socket management, and authentication functionalities. Supporting various connection types, including TCP, SSL, and Unix domain sockets, it also implements retry mechanisms for handling connection errors. This component ensures reliable and efficient communication with the Redis server. +- **Related Classes/Methods**: `redis.connection.ConnectionPool` (1309:1654), `redis.asyncio.connection.ConnectionPool` (1031:1253), `redis.connection.Connection` (730:801), `redis.asyncio.connection.Connection` (723:777) + +### Cluster Support +The Cluster Support component provides functionalities for managing and interacting with Redis clusters, handling node discovery, slot assignment, and command routing. It supports both synchronous and asynchronous cluster operations, offering a consistent API for interacting with Redis clusters. This component is responsible for distributing commands across the cluster and handling failover scenarios, ensuring high availability and scalability. +- **Related Classes/Methods**: `redis.cluster.RedisCluster` (456:1360), `redis.asyncio.cluster.RedisCluster` (99:989), `redis.cluster.NodesManager` (1443:1863), `redis.asyncio.cluster.NodesManager` (1211:1518) + +### PubSub Management +The PubSub Management component provides publish/subscribe functionalities for real-time messaging, enabling clients to subscribe to channels and receive messages published to those channels. It supports both synchronous and asynchronous pub/sub operations, managing subscriptions and distributing messages to subscribers efficiently. This component facilitates real-time communication and event-driven architectures. +- **Related Classes/Methods**: `redis.client.PubSub` (743:1241), `redis.asyncio.client.PubSub` (803:1231), `redis.cluster.ClusterPubSub` (1866:2107) + +### Data Handling +The Data Handling component manages the serialization of commands and parsing of responses between the client and the Redis server. It supports different serialization formats, including RESP2 and RESP3, and includes helper functions for parsing specific data types. This component ensures that data is correctly formatted for transmission to and from the Redis server, maintaining data integrity and compatibility. +- **Related Classes/Methods**: `redis._parsers.encoders.Encoder` (4:44), `redis._parsers.commands.CommandsParser` (56:170), `redis._parsers.commands.AsyncCommandsParser` (173:281), `redis._parsers.helpers` (full file reference) \ No newline at end of file From 32bae8a55472d49c53169fa932cdfb688fdcb1c2 Mon Sep 17 00:00:00 2001 From: ivanmilevtues Date: Tue, 10 Jun 2025 01:55:17 +0200 Subject: [PATCH 2/3] Added latest generation for onboarding docs --- .codeboarding/Command & Module Execution.md | 190 ++++++++++++++++++ .../Connection & Protocol Management.md | 174 ++++++++++++++++ .codeboarding/Error Handling.md | 186 +++++++++++++++++ .../High Availability & Cluster Management.md | 153 ++++++++++++++ .codeboarding/Redis Client Core.md | 120 +++++++++++ .codeboarding/on_boarding.md | 87 ++++++++ CodeBoarding/Client Interface.md | 52 ----- CodeBoarding/Cluster Support.md | 53 ----- CodeBoarding/Command Abstraction.md | 66 ------ CodeBoarding/Connection Management.md | 66 ------ CodeBoarding/Data Handling.md | 54 ----- CodeBoarding/PubSub Management.md | 43 ---- CodeBoarding/on_boarding.md | 50 ----- 13 files changed, 910 insertions(+), 384 deletions(-) create mode 100644 .codeboarding/Command & Module Execution.md create mode 100644 .codeboarding/Connection & Protocol Management.md create mode 100644 .codeboarding/Error Handling.md create mode 100644 .codeboarding/High Availability & Cluster Management.md create mode 100644 .codeboarding/Redis Client Core.md create mode 100644 .codeboarding/on_boarding.md delete mode 100644 CodeBoarding/Client Interface.md delete mode 100644 CodeBoarding/Cluster Support.md delete mode 100644 CodeBoarding/Command Abstraction.md delete mode 100644 CodeBoarding/Connection Management.md delete mode 100644 CodeBoarding/Data Handling.md delete mode 100644 CodeBoarding/PubSub Management.md delete mode 100644 CodeBoarding/on_boarding.md diff --git a/.codeboarding/Command & Module Execution.md b/.codeboarding/Command & Module Execution.md new file mode 100644 index 0000000000..3d84c2e47f --- /dev/null +++ b/.codeboarding/Command & Module Execution.md @@ -0,0 +1,190 @@ +```mermaid +graph LR + RedisCoreCommands["RedisCoreCommands"] + RedisModuleIntegration["RedisModuleIntegration"] + RedisJSONModule["RedisJSONModule"] + RedisSearchModule["RedisSearchModule"] + RedisTimeSeriesModule["RedisTimeSeriesModule"] + RedisBloomModule["RedisBloomModule"] + RedisVectorSetModule["RedisVectorSetModule"] + RedisClusterManagement["RedisClusterManagement"] + RedisScriptingAndPubSub["RedisScriptingAndPubSub"] + RedisCommandHelpers["RedisCommandHelpers"] + RedisModuleIntegration -- "integrates" --> RedisJSONModule + RedisModuleIntegration -- "integrates" --> RedisSearchModule + RedisModuleIntegration -- "integrates" --> RedisTimeSeriesModule + RedisModuleIntegration -- "integrates" --> RedisBloomModule + RedisModuleIntegration -- "integrates" --> RedisVectorSetModule + RedisClusterManagement -- "utilizes" --> RedisCommandHelpers + RedisClusterManagement -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisCoreCommands -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisCoreCommands -- "utilizes" --> RedisCommandHelpers + RedisScriptingAndPubSub -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisScriptingAndPubSub -- "utilizes" --> RedisCommandHelpers + RedisVectorSetModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisVectorSetModule -- "utilizes" --> RedisCommandHelpers + RedisTimeSeriesModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisTimeSeriesModule -- "utilizes" --> RedisCommandHelpers + RedisSearchModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisSearchModule -- "utilizes" --> RedisCommandHelpers + RedisBloomModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisBloomModule -- "utilizes" --> RedisCommandHelpers + RedisJSONModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisJSONModule -- "utilizes" --> RedisCommandHelpers +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + +## Component Details + +This component overview describes the structure, flow, and purpose of the Command & Module Execution subsystem within the Redis client library. It focuses on how standard Redis commands are implemented and executed, and how interactions with various Redis Modules (e.g., JSON, Search, TimeSeries) are facilitated. The system provides a comprehensive interface for both core Redis functionalities and extended module capabilities, ensuring efficient data operations and server management. + +### RedisCoreCommands +This component provides the fundamental Redis commands for interacting with various data structures (strings, lists, sets, sorted sets, hashes, streams, geo-spatial) and handles administrative and management commands for the Redis server (configuration, client management, server-wide operations, ACL, modules, cluster commands). It serves as the primary interface for basic Redis operations. + + +**Related Classes/Methods**: + +- `redis.commands.core.ACLCommands` (full file reference) +- `redis.commands.core.ManagementCommands` (full file reference) +- `redis.commands.core.AsyncManagementCommands` (full file reference) +- `redis.commands.core.BitFieldOperation` (full file reference) +- `redis.commands.core.BasicKeyCommands` (full file reference) +- `redis.commands.core.AsyncBasicKeyCommands` (full file reference) +- `redis.commands.core.ListCommands` (full file reference) +- `redis.commands.core.ScanCommands` (full file reference) +- `redis.commands.core.AsyncScanCommands` (full file reference) +- `redis.commands.core.SetCommands` (full file reference) +- `redis.commands.core.StreamCommands` (full file reference) +- `redis.commands.core.SortedSetCommands` (full file reference) +- `redis.commands.core.HyperlogCommands` (full file reference) +- `redis.commands.core.HashCommands` (full file reference) +- `redis.commands.core.GeoCommands` (full file reference) +- `redis.commands.core.ModuleCommands` (full file reference) +- `redis.commands.core.AsyncModuleCommands` (full file reference) +- `redis.commands.core.ClusterCommands` (full file reference) + + +### RedisModuleIntegration +This component acts as an integration layer for various Redis modules, providing a unified interface to access their functionalities. It serves as a central point for dispatching commands to specific module clients. + + +**Related Classes/Methods**: + +- `redis.commands.redismodules.RedisModuleCommands` (14:91) +- `redis.commands.redismodules.AsyncRedisModuleCommands` (94:101) + + +### RedisJSONModule +This component offers a client for the RedisJSON module, allowing for efficient storage and manipulation of JSON documents within Redis. It provides methods for JSON-specific operations like setting, getting, and manipulating JSON paths. + + +**Related Classes/Methods**: + +- `redis.commands.json.commands.JSONCommands` (13:431) +- `redis.commands.json.JSON` (full file reference) + + +### RedisSearchModule +This component provides a comprehensive client for the Redis Search module, supporting index creation, document management, complex query building, and aggregation. It enables full-text search capabilities within Redis. + + +**Related Classes/Methods**: + +- `redis.commands.search.index_definition.IndexDefinition` (11:79) +- `redis.commands.search.aggregation.AggregateRequest` (89:372) +- `redis.commands.search.field.TextField` (79:109) +- `redis.commands.search.field.NumericField` (112:118) +- `redis.commands.search.field.GeoShapeField` (121:133) +- `redis.commands.search.field.GeoField` (136:142) +- `redis.commands.search.field.TagField` (145:168) +- `redis.commands.search.field.VectorField` (171:210) +- `redis.commands.search.commands.SearchCommands` (full file reference) +- `redis.commands.search.commands.AsyncSearchCommands` (full file reference) +- `redis.commands.search.query.Query` (6:339) +- `redis.commands.search.query.NumericFilter` (347:364) +- `redis.commands.search.query.GeoFilter` (367:376) +- `redis.commands.search.querystring` (316:317) +- `redis.commands.search.reducers` (full file reference) +- `redis.commands.search.suggestion.Suggestion` (6:20) +- `redis.commands.search.suggestion.SuggestionParser` (23:55) +- `redis.commands.search.result.Result` (7:87) +- `redis.commands.search.Search` (full file reference) +- `redis.commands.search.AsyncSearch` (full file reference) + + +### RedisTimeSeriesModule +This component offers a client for the Redis TimeSeries module, allowing for the creation, manipulation, and querying of time-series data. It provides functionalities for adding samples, querying ranges, and managing time-series data. + + +**Related Classes/Methods**: + +- `redis.commands.timeseries.info.TSInfo` (5:91) +- `redis.commands.timeseries.commands.TimeSeriesCommands` (25:1000) +- `redis.commands.timeseries.TimeSeries` (full file reference) + + +### RedisBloomModule +This component provides client-side access to the RedisBloom module, enabling the use of probabilistic data structures like Bloom filters, Cuckoo filters, Count-Min sketches, and TopK. It offers methods for interacting with these specialized data structures. + + +**Related Classes/Methods**: + +- `redis.commands.bf.info.BFInfo` (4:26) +- `redis.commands.bf.info.CFInfo` (29:57) +- `redis.commands.bf.info.TDigestInfo` (92:120) +- `redis.commands.bf.commands.TOPKCommands` (292:356) +- `redis.commands.bf.CMSBloom` (full file reference) +- `redis.commands.bf.TOPKBloom` (full file reference) +- `redis.commands.bf.CFBloom` (full file reference) +- `redis.commands.bf.TDigestBloom` (full file reference) +- `redis.commands.bf.BFBloom` (full file reference) + + +### RedisVectorSetModule +This component provides the client-side interface for interacting with the Redis VectorSet module, enabling vector similarity search and related operations. It allows for storing and querying vector data within Redis. + + +**Related Classes/Methods**: + +- `redis.commands.vectorset.commands.VectorSetCommands` (40:367) +- `redis.commands.vectorset.VectorSet` (full file reference) + + +### RedisClusterManagement +This component is responsible for managing Redis Cluster specific operations, including multi-key commands, node management, and data partitioning across slots. It provides an interface for interacting with a Redis Cluster setup. + + +**Related Classes/Methods**: + +- `redis.commands.cluster.ClusterMultiKeyCommands` (99:260) +- `redis.commands.cluster.AsyncClusterMultiKeyCommands` (263:339) +- `redis.commands.cluster.ClusterManagementCommands` (342:692) +- `redis.commands.cluster.AsyncClusterManagementCommands` (695:719) +- `redis.commands.cluster.ClusterDataAccessCommands` (722:810) + + +### RedisScriptingAndPubSub +This component encapsulates functionalities related to Redis scripting (Lua scripts) and Publish/Subscribe messaging. It provides methods for executing scripts, managing script caches, and handling pub/sub operations. + + +**Related Classes/Methods**: + +- `redis.commands.core.Script` (full file reference) +- `redis.commands.core.PubSubCommands` (full file reference) +- `redis.commands.core.ScriptCommands` (full file reference) +- `redis.commands.core.AsyncScriptCommands` (full file reference) +- `redis.commands.core.FunctionCommands` (full file reference) + + +### RedisCommandHelpers +This component provides various utility functions and helper methods that are commonly used across different Redis command implementations for tasks like argument parsing, data conversion, and protocol version handling. It supports the other command components by offering shared functionalities. + + +**Related Classes/Methods**: + +- `redis.commands.helpers` (full file reference) + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Connection & Protocol Management.md b/.codeboarding/Connection & Protocol Management.md new file mode 100644 index 0000000000..e40d2710f5 --- /dev/null +++ b/.codeboarding/Connection & Protocol Management.md @@ -0,0 +1,174 @@ +```mermaid +graph LR + Core_Connection_Management["Core Connection Management"] + Connection_Pooling["Connection Pooling"] + Protocol_Parsers["Protocol Parsers"] + Authentication_Security["Authentication & Security"] + Error_Handling["Error Handling"] + Client_Interfaces["Client Interfaces"] + Utility_Eventing["Utility & Eventing"] + Retry_Backoff["Retry & Backoff"] + Caching["Caching"] + Client_Interfaces -- "uses" --> Connection_Pooling + Connection_Pooling -- "manages" --> Core_Connection_Management + Core_Connection_Management -- "uses" --> Protocol_Parsers + Core_Connection_Management -- "raises" --> Error_Handling + Core_Connection_Management -- "integrates" --> Authentication_Security + Core_Connection_Management -- "applies" --> Retry_Backoff + Protocol_Parsers -- "raises" --> Error_Handling + Authentication_Security -- "raises" --> Error_Handling + Connection_Pooling -- "notifies" --> Utility_Eventing + Connection_Pooling -- "leverages" --> Caching + Client_Interfaces -- "utilizes" --> Caching + Utility_Eventing -- "supports" --> Core_Connection_Management + Utility_Eventing -- "supports" --> Protocol_Parsers +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + +## Component Details + +The Connection & Protocol Management subsystem in the Redis client library is a comprehensive system designed to facilitate robust and efficient communication with Redis servers. It manages the entire lifecycle of connections, from establishment and pooling to secure authentication and error handling. The subsystem handles the intricacies of Redis Serialization Protocols (RESP2 and RESP3) for command encoding and response parsing, ensuring data integrity and performance. It also incorporates caching mechanisms, retry strategies for transient failures, and provides high-level client interfaces that abstract the underlying complexities, enabling seamless interaction for users. + +### Core Connection Management +This component encapsulates the fundamental logic for establishing, maintaining, and managing connections to Redis servers, including both synchronous and asynchronous implementations. It handles various connection types such as standard TCP, SSL, and Unix domain sockets, and manages the lifecycle of individual connections. + + +**Related Classes/Methods**: + +- `redis.connection.AbstractConnection` (full file reference) +- `redis.connection.Connection` (full file reference) +- `redis.connection.SSLConnection` (full file reference) +- `redis.connection.UnixDomainSocketConnection` (full file reference) +- `redis.connection.CacheProxyConnection` (full file reference) +- `redis.asyncio.connection.AbstractConnection` (106:720) +- `redis.asyncio.connection.Connection` (723:777) +- `redis.asyncio.connection.SSLConnection` (780:844) +- `redis.asyncio.connection.UnixDomainSocketConnection` (916:937) + + +### Connection Pooling +This component is responsible for efficiently managing a pool of Redis connections. It optimizes resource utilization by reusing existing connections, thereby reducing the overhead of establishing new connections for each operation. It provides mechanisms for acquiring and releasing connections, and supports both blocking and non-blocking behaviors. + + +**Related Classes/Methods**: + +- `redis.connection.ConnectionPool` (full file reference) +- `redis.connection.BlockingConnectionPool` (full file reference) +- `redis.asyncio.connection.ConnectionPool` (full file reference) +- `redis.asyncio.connection.BlockingConnectionPool` (full file reference) + + +### Protocol Parsers +This component handles the serialization of commands sent to Redis and the deserialization of responses received from the server. It supports different Redis Serialization Protocols (RESP2 and RESP3) and can leverage optimized C implementations like hiredis for improved performance. It also includes logic for parsing command structures and handling socket buffer operations. + + +**Related Classes/Methods**: + +- `redis._parsers.encoders.Encoder` (4:44) +- `redis._parsers.socket.SocketBuffer` (29:162) +- `redis._parsers.commands.AbstractCommandsParser` (10:53) +- `redis._parsers.commands.CommandsParser` (56:170) +- `redis._parsers.commands.AsyncCommandsParser` (173:281) +- `redis._parsers.hiredis._HiredisParser` (41:184) +- `redis._parsers.hiredis._AsyncHiredisParser` (187:295) +- `redis._parsers.resp3._RESP3Parser` (15:131) +- `redis._parsers.resp3._AsyncRESP3Parser` (134:257) +- `redis._parsers.base.BaseParser` (54:105) +- `redis._parsers.base._RESPBase` (108:140) +- `redis._parsers.base._AsyncRESPBase` (213:289) +- `redis._parsers.resp2._RESP2Parser` (9:68) +- `redis._parsers.resp2._AsyncRESP2Parser` (71:132) + + +### Authentication & Security +This component provides functionalities for authenticating with Redis servers and ensuring secure communication. It includes mechanisms for managing authentication tokens, such as JWT, and implements OCSP (Online Certificate Status Protocol) verification to validate SSL certificates, enhancing the security posture of connections. + + +**Related Classes/Methods**: + +- `redis.auth.token_manager.TokenManager` (121:340) +- `redis.auth.token.SimpleToken` (44:75) +- `redis.auth.token.JWToken` (78:130) +- `redis.ocsp._verify_response` (22:47) +- `redis.ocsp._check_certificate` (50:106) +- `redis.ocsp._get_certificates` (109:123) +- `redis.ocsp.ocsp_staple_verifier` (142:167) +- `redis.ocsp.OCSPVerifier` (170:308) +- `redis.asyncio.connection.RedisSSLContext` (847:913) + + +### Error Handling +This component defines a comprehensive set of custom exception classes that represent various error conditions encountered during Redis operations, such as connection failures, timeouts, authentication issues, and invalid responses. It centralizes error management, allowing for more specific and robust error handling throughout the library. + + +**Related Classes/Methods**: + +- `redis.exceptions.DataError` (36:37) +- `redis.exceptions.ConnectionError` (8:9) +- `redis.exceptions.TimeoutError` (12:13) +- `redis.exceptions.AuthenticationError` (16:17) +- `redis.exceptions.RedisError` (4:5) +- `redis.exceptions.InvalidResponse` (28:29) +- `redis.exceptions.ResponseError` (32:33) +- `redis.auth.err.TokenRenewalErr` (25:31) +- `redis.auth.err.InvalidTokenSchemaErr` (13:22) +- `redis.exceptions.AuthorizationError` (20:21) + + +### Client Interfaces +This component provides the high-level API for users to interact with Redis. It includes the standard Redis client, as well as specialized clients for Redis Cluster and Redis Sentinel, abstracting the underlying connection and protocol details to offer a user-friendly interface for executing Redis commands. + + +**Related Classes/Methods**: + +- `redis.cluster.RedisCluster` (456:1000) +- `redis.cluster.NodesManager` (full file reference) +- `redis.sentinel.SentinelConnectionPool` (137:195) +- `redis.client.Redis` (112:670) + + +### Utility & Eventing +This component comprises a collection of general-purpose utility functions that support various operations across the library, such as string manipulation, version comparison, and argument deprecation handling. It also includes an event dispatching mechanism that allows different parts of the system to communicate and react to specific events, like connection releases. + + +**Related Classes/Methods**: + +- `redis.utils.get_lib_version` (211:216) +- `redis.utils.format_error_message` (219:228) +- `redis.utils.str_if_bytes` (60:63) +- `redis.utils.deprecated_args` (153:195) +- `redis.utils.ensure_string` (261:267) +- `redis.utils.compare_versions` (231:258) +- `redis.event.EventDispatcher` (57:90) +- `redis.event.AfterConnectionReleasedEvent` (93:103) +- `redis.event.AsyncAfterConnectionReleasedEvent` (106:107) + + +### Retry & Backoff +This component implements robust strategies for handling transient failures by retrying operations with configurable delays. It includes mechanisms for exponential backoff and allows for the definition of supported error types, ensuring that the client can gracefully recover from temporary network issues or server unavailability. + + +**Related Classes/Methods**: + +- `redis.retry.Retry` (13:95) +- `redis.backoff.NoBackoff` (47:51) +- `redis.asyncio.retry.Retry` (13:79) + + +### Caching +This component provides an internal caching mechanism used to store and retrieve frequently accessed data, such as command information or connection details, to improve performance. It includes different caching policies, like LRU, and a factory for creating cache instances. + + +**Related Classes/Methods**: + +- `redis.cache.DefaultCache` (142:224) +- `redis.cache.LRUPolicy` (227:271) +- `redis.cache.CacheFactory` (392:401) +- `redis.cache.CacheConfig` (278:383) +- `redis.cache.CacheKey` (19:21) +- `redis.cache.CacheEntry` (24:43) + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Error Handling.md b/.codeboarding/Error Handling.md new file mode 100644 index 0000000000..8befc06fa6 --- /dev/null +++ b/.codeboarding/Error Handling.md @@ -0,0 +1,186 @@ +```mermaid +graph LR + Error_Handling["Error Handling"] + Redis_Client["Redis Client"] + Connection_Management["Connection Management"] + Redis_Cluster_Management["Redis Cluster Management"] + Distributed_Locking["Distributed Locking"] + Redis_Sentinel_Integration["Redis Sentinel Integration"] + Protocol_Parsing["Protocol Parsing"] + Authentication_and_Security["Authentication and Security"] + Redis_Client -- "utilizes" --> Connection_Management + Redis_Client -- "interprets responses via" --> Protocol_Parsing + Redis_Client -- "raises" --> Error_Handling + Connection_Management -- "relies on" --> Protocol_Parsing + Connection_Management -- "raises" --> Error_Handling + Redis_Cluster_Management -- "manages connections through" --> Connection_Management + Redis_Cluster_Management -- "raises" --> Error_Handling + Distributed_Locking -- "leverages" --> Redis_Client + Distributed_Locking -- "raises" --> Error_Handling + Redis_Sentinel_Integration -- "manages connections through" --> Connection_Management + Redis_Sentinel_Integration -- "raises" --> Error_Handling + Protocol_Parsing -- "raises" --> Error_Handling + Authentication_and_Security -- "configures" --> Connection_Management +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + +## Component Details + +This architecture overview describes the core components of the redis-py library, focusing on how it manages connections, handles Redis commands, interacts with Redis clusters and Sentinels, and provides robust error handling and security features. The main flow involves clients initiating connections through connection management, sending commands which are then parsed and executed, and receiving responses that are processed by the protocol parsing component. Error handling is integrated throughout, providing specific exceptions for various operational failures. Asynchronous capabilities are seamlessly integrated into the core components, offering non-blocking operations for high-performance applications. + +### Error Handling +Defines and manages custom exception classes for various Redis-related errors, providing a structured and specific way to handle different error scenarios that can occur during client-server interactions. + + +**Related Classes/Methods**: + +- `redis.exceptions.RedisError` (4:5) +- `redis.exceptions.ConnectionError` (8:9) +- `redis.exceptions.AuthenticationError` (16:17) +- `redis.exceptions.AuthorizationError` (20:21) +- `redis.exceptions.BusyLoadingError` (24:25) +- `redis.exceptions.MaxConnectionsError` (223:223) +- `redis.exceptions.TimeoutError` (12:13) +- `redis.exceptions.DataError` (36:37) +- `redis.exceptions.PubSubError` (40:41) +- `redis.exceptions.InvalidResponse` (28:29) +- `redis.exceptions.ResponseError` (32:33) +- `redis.exceptions.AuthenticationWrongNumberOfArgsError` (103:109) +- `redis.exceptions.NoPermissionError` (72:73) +- `redis.exceptions.ExecAbortError` (64:65) +- `redis.exceptions.ReadOnlyError` (68:69) +- `redis.exceptions.NoScriptError` (48:49) +- `redis.exceptions.OutOfMemoryError` (52:61) +- `redis.exceptions.ModuleError` (76:77) +- `redis.exceptions.WatchError` (44:45) +- `redis.exceptions.LockError` (80:88) +- `redis.exceptions.LockNotOwnedError` (91:94) +- `redis.exceptions.TryAgainError` (171:179) +- `redis.exceptions.ClusterError` (120:126) +- `redis.exceptions.ClusterDownError` (129:142) +- `redis.exceptions.MasterDownError` (202:208) +- `redis.exceptions.AskError` (145:168) +- `redis.exceptions.MovedError` (192:199) +- `redis.exceptions.ClusterCrossSlotError` (182:189) +- `redis.exceptions.RedisClusterException` (112:117) +- `redis.exceptions.SlotNotCoveredError` (211:220) +- `redis.exceptions.CrossSlotTransactionError` (226:232) +- `redis.exceptions.InvalidPipelineStack` (235:241) + + +### Redis Client +Provides the fundamental synchronous and asynchronous interfaces for interacting with a single Redis instance. It includes functionalities for executing commands, managing Pub/Sub subscriptions, and handling transactional pipelines. + + +**Related Classes/Methods**: + +- `redis.client.Redis` (112:670) +- `redis.client.Monitor` (676:740) +- `redis.client.PubSub` (743:1000) +- `redis.client.Pipeline` (full file reference) +- `redis.asyncio.client.Redis` (full file reference) +- `redis.asyncio.client.Monitor` (full file reference) +- `redis.asyncio.client.PubSub` (full file reference) +- `redis.asyncio.client.Pipeline` (full file reference) + + +### Connection Management +Responsible for establishing, maintaining, and pooling synchronous and asynchronous connections to Redis servers. It handles connection health checks, error handling during connection attempts, and SSL/TLS wrapping. + + +**Related Classes/Methods**: + +- `redis.connection.HiredisRespSerializer` (full file reference) +- `redis.connection.AbstractConnection` (full file reference) +- `redis.connection.CacheProxyConnection` (full file reference) +- `redis.connection.SSLConnection` (full file reference) +- `redis.connection.ConnectionPool` (full file reference) +- `redis.connection.BlockingConnectionPool` (full file reference) +- `redis.asyncio.connection.AbstractConnection` (106:720) +- `redis.asyncio.connection.SSLConnection` (780:844) +- `redis.asyncio.connection.RedisSSLContext` (847:913) +- `redis.asyncio.connection.ConnectionPool` (full file reference) +- `redis.asyncio.connection.BlockingConnectionPool` (full file reference) + + +### Redis Cluster Management +Manages interactions with a Redis Cluster for both synchronous and asynchronous operations. It handles node discovery, slot mapping, command routing, and error handling specific to a clustered environment, including transaction and pipeline strategies within the cluster. + + +**Related Classes/Methods**: + +- `redis.cluster.RedisCluster` (456:1000) +- `redis.cluster.NodesManager` (full file reference) +- `redis.cluster.ClusterPubSub` (full file reference) +- `redis.cluster.AbstractStrategy` (full file reference) +- `redis.cluster.PipelineStrategy` (full file reference) +- `redis.cluster.TransactionStrategy` (full file reference) +- `redis.commands.cluster.ClusterManagementCommands` (342:692) +- `redis.asyncio.cluster.RedisCluster` (full file reference) +- `redis.asyncio.cluster.ClusterNode` (full file reference) +- `redis.asyncio.cluster.NodesManager` (full file reference) +- `redis.asyncio.cluster.PipelineStrategy` (full file reference) +- `redis.asyncio.cluster.TransactionStrategy` (full file reference) + + +### Distributed Locking +Provides mechanisms for implementing distributed locks using Redis, ensuring atomicity and proper release of locks for both synchronous and asynchronous contexts. + + +**Related Classes/Methods**: + +- `redis.lock.Lock` (14:343) +- `redis.asyncio.lock.Lock` (17:334) + + +### Redis Sentinel Integration +Facilitates interaction with Redis Sentinel for high availability, allowing clients to discover master and slave nodes and handle failovers for both synchronous and asynchronous operations. + + +**Related Classes/Methods**: + +- `redis.sentinel.SentinelManagedConnection` (20:82) +- `redis.sentinel.SentinelConnectionPoolProxy` (89:134) +- `redis.sentinel.Sentinel` (198:410) +- `redis.asyncio.sentinel.SentinelManagedConnection` (26:89) +- `redis.asyncio.sentinel.SentinelConnectionPool` (96:164) +- `redis.asyncio.sentinel.Sentinel` (167:389) + + +### Protocol Parsing +Responsible for parsing responses from the Redis server according to the RESP protocol, including handling different RESP versions and error responses for both synchronous and asynchronous contexts. + + +**Related Classes/Methods**: + +- `redis._parsers.encoders.Encoder` (4:44) +- `redis._parsers.socket.SocketBuffer` (29:162) +- `redis._parsers.commands.CommandsParser` (56:170) +- `redis._parsers.commands.AsyncCommandsParser` (173:281) +- `redis._parsers.hiredis._HiredisParser` (41:184) +- `redis._parsers.hiredis._AsyncHiredisParser` (187:295) +- `redis._parsers.resp3._RESP3Parser` (15:131) +- `redis._parsers.resp3._AsyncRESP3Parser` (134:257) +- `redis._parsers.base.BaseParser` (54:105) +- `redis._parsers.base._AsyncRESPBase` (213:289) +- `redis._parsers.resp2._RESP2Parser` (9:68) +- `redis._parsers.resp2._AsyncRESP2Parser` (71:132) + + +### Authentication and Security +Handles authentication mechanisms, including token management and OCSP verification for secure connections. + + +**Related Classes/Methods**: + +- `redis.ocsp._verify_response` (22:47) +- `redis.ocsp._check_certificate` (50:106) +- `redis.ocsp.ocsp_staple_verifier` (142:167) +- `redis.ocsp.OCSPVerifier` (170:308) +- `redis.auth.token_manager.TokenManager` (121:340) +- `redis.auth.token.JWToken` (78:130) + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/High Availability & Cluster Management.md b/.codeboarding/High Availability & Cluster Management.md new file mode 100644 index 0000000000..b972d6c24f --- /dev/null +++ b/.codeboarding/High Availability & Cluster Management.md @@ -0,0 +1,153 @@ +```mermaid +graph LR + Redis_Cluster_Client["Redis Cluster Client"] + Cluster_Node_Manager["Cluster Node Manager"] + Cluster_Node_Representation["Cluster Node Representation"] + Load_Balancer["Load Balancer"] + Cluster_Pub_Sub["Cluster Pub/Sub"] + Cluster_Pipeline["Cluster Pipeline"] + Pipeline_Strategy["Pipeline Strategy"] + Transaction_Strategy["Transaction Strategy"] + Redis_Sentinel_Client["Redis Sentinel Client"] + Sentinel_Connection_Pool["Sentinel Connection Pool"] + Redis_Cluster_Client -- "manages" --> Cluster_Node_Manager + Redis_Cluster_Client -- "creates" --> Cluster_Pipeline + Redis_Cluster_Client -- "creates" --> Cluster_Pub_Sub + Cluster_Node_Manager -- "provides nodes to" --> Redis_Cluster_Client + Cluster_Node_Manager -- "manages" --> Cluster_Node_Representation + Cluster_Node_Manager -- "uses" --> Load_Balancer + Cluster_Pipeline -- "delegates execution to" --> Pipeline_Strategy + Cluster_Pipeline -- "delegates execution to" --> Transaction_Strategy + Redis_Sentinel_Client -- "uses" --> Sentinel_Connection_Pool + Sentinel_Connection_Pool -- "provides connections to" --> Redis_Sentinel_Client +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + +## Component Details + +This graph illustrates the architecture of the High Availability & Cluster Management subsystem, focusing on how Redis Cluster and Redis Sentinel functionalities are provided. It details the components responsible for node discovery, command routing, pipeline execution, and high-availability management in both cluster and sentinel environments. + +### Redis Cluster Client +The Redis Cluster Client (`RedisCluster`) serves as the primary interface for applications to interact with a Redis Cluster. It handles command routing to appropriate nodes, manages retries on MOVED/ASK errors, and provides methods for obtaining pipeline and pub/sub instances. It supports both synchronous and asynchronous operations. + + +**Related Classes/Methods**: + +- `redis.cluster.RedisCluster` (456:1000) +- `redis.cluster.RedisCluster:__init__` (513:712) +- `redis.cluster.RedisCluster:execute_command` (full file reference) +- `redis.cluster.RedisCluster:pipeline` (841:866) +- `redis.cluster.RedisCluster:pubsub` (834:839) +- `redis.asyncio.cluster.RedisCluster:__init__` (full file reference) +- `redis.asyncio.cluster.RedisCluster:execute_command` (full file reference) +- `redis.asyncio.cluster.RedisCluster:pipeline` (full file reference) + + +### Cluster Node Manager +The Cluster Node Manager (`NodesManager`) is responsible for discovering and maintaining the topology of the Redis Cluster. It manages the cache of active nodes and their assigned slots, handles updates for MOVED exceptions, and initializes Redis connections to cluster nodes. It also integrates with a load balancer for read operations. + + +**Related Classes/Methods**: + +- `redis.cluster.NodesManager:__init__` (full file reference) +- `redis.cluster.NodesManager:initialize` (full file reference) +- `redis.cluster.NodesManager:get_node_from_slot` (full file reference) +- `redis.asyncio.cluster.NodesManager:__init__` (full file reference) +- `redis.asyncio.cluster.NodesManager:initialize` (full file reference) +- `redis.asyncio.cluster.NodesManager:get_node_from_slot` (full file reference) + + +### Cluster Node Representation +The Cluster Node Representation (`ClusterNode`) encapsulates the details of a single Redis instance within the cluster, including its host, port, name, and server type (primary or replica). It also holds a reference to the Redis connection object for that specific node. + + +**Related Classes/Methods**: + +- `redis.cluster.ClusterNode:__init__` (full file reference) +- `redis.asyncio.cluster.ClusterNode:__init__` (full file reference) + + +### Load Balancer +The Load Balancer (`LoadBalancer`) provides strategies for distributing read requests across multiple replica nodes associated with a primary. It supports round-robin and random replica selection to optimize read performance and distribute load. + + +**Related Classes/Methods**: + +- `redis.cluster.LoadBalancer:get_server_index` (full file reference) + + +### Cluster Pub/Sub +The Cluster Pub/Sub (`ClusterPubSub`) component extends the standard Redis Pub/Sub functionality to work within a Redis Cluster environment. It manages the pub/sub connection to a specific cluster node, determined by channel keyslots or a random node. + + +**Related Classes/Methods**: + +- `redis.cluster.ClusterPubSub:__init__` (full file reference) +- `redis.cluster.ClusterPubSub:execute_command` (full file reference) +- `redis.asyncio.cluster.ClusterPubSub:__init__` (full file reference) + + +### Cluster Pipeline +The Cluster Pipeline (`ClusterPipeline`) enables batching of multiple Redis commands for efficient execution within a Redis Cluster. It collects commands and then delegates their execution to specific strategies (PipelineStrategy or TransactionStrategy) based on whether a transaction is involved. + + +**Related Classes/Methods**: + +- `redis.cluster.ClusterPipeline:__init__` (full file reference) +- `redis.cluster.ClusterPipeline:execute` (full file reference) +- `redis.asyncio.cluster.ClusterPipeline:__init__` (full file reference) +- `redis.asyncio.cluster.ClusterPipeline:execute` (full file reference) + + +### Pipeline Strategy +The Pipeline Strategy (`PipelineStrategy`) defines how a batch of commands collected by the Cluster Pipeline is executed across the Redis Cluster. It handles routing commands to the correct nodes and processing their responses, without transactional guarantees. + + +**Related Classes/Methods**: + +- `redis.cluster.PipelineStrategy:__init__` (full file reference) +- `redis.cluster.PipelineStrategy:execute` (full file reference) +- `redis.asyncio.cluster.PipelineStrategy:__init__` (full file reference) +- `redis.asyncio.cluster.PipelineStrategy:execute` (full file reference) + + +### Transaction Strategy +The Transaction Strategy (`TransactionStrategy`) implements the logic for executing Redis transactions (MULTI/EXEC) within a Redis Cluster. It ensures that all commands within a transaction are sent to the same node and handles WATCH errors and retries. + + +**Related Classes/Methods**: + +- `redis.cluster.TransactionStrategy:__init__` (full file reference) +- `redis.cluster.TransactionStrategy:execute` (full file reference) +- `redis.asyncio.cluster.TransactionStrategy:__init__` (full file reference) +- `redis.asyncio.cluster.TransactionStrategy:execute` (full file reference) + + +### Redis Sentinel Client +The Redis Sentinel Client (`Sentinel`) provides an interface for applications to connect to Redis instances managed by Sentinel for high availability. It discovers the current master and available replicas for a given service name. + + +**Related Classes/Methods**: + +- `redis.sentinel.Sentinel` (198:410) +- `redis.sentinel.Sentinel:__init__` (227:249) +- `redis.sentinel.Sentinel:master_for` (343:379) +- `redis.asyncio.sentinel.Sentinel:__init__` (196:218) +- `redis.asyncio.sentinel.Sentinel:master_for` (320:357) + + +### Sentinel Connection Pool +The Sentinel Connection Pool (`SentinelConnectionPool` and `SentinelManagedConnection`) manages the underlying network connections to Redis master and replica instances discovered via Sentinel. It ensures efficient connection reuse and handles connection lifecycle. + + +**Related Classes/Methods**: + +- `redis.sentinel.SentinelConnectionPool:__init__` (145:166) +- `redis.sentinel.SentinelManagedConnection:__init__` (21:23) +- `redis.asyncio.sentinel.SentinelConnectionPool:__init__` (104:120) +- `redis.asyncio.sentinel.SentinelManagedConnection:__init__` (27:29) + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Redis Client Core.md b/.codeboarding/Redis Client Core.md new file mode 100644 index 0000000000..7661b5a5e0 --- /dev/null +++ b/.codeboarding/Redis Client Core.md @@ -0,0 +1,120 @@ +```mermaid +graph LR + Redis_Client_Core["Redis Client Core"] + Lock_Management["Lock Management"] + Pub_Sub_Component["Pub/Sub Component"] + Pipeline_Transaction_Component["Pipeline/Transaction Component"] + Redis_Client_Core -- "creates instances of" --> Pub_Sub_Component + Redis_Client_Core -- "creates instances of" --> Pipeline_Transaction_Component + Lock_Management -- "utilizes" --> Redis_Client_Core + Pipeline_Transaction_Component -- "extends functionality of" --> Redis_Client_Core +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + +## Component Details + +This graph illustrates the core components of the Redis client library, focusing on the primary interface for Redis interaction, `Redis Client Core`, and its specialized functionalities. It details how `Redis Client Core` orchestrates operations, including the creation of `Pub/Sub Component` for real-time messaging and `Pipeline/Transaction Component` for efficient batched commands. Additionally, it shows how `Lock Management` leverages the `Redis Client Core` for distributed locking, and how the `Pipeline/Transaction Component` builds upon the core client's capabilities. + +### Redis Client Core +The Redis Client Core component serves as the primary interface for interacting with a Redis server. It manages connection pools, executes commands, handles command parsing, and provides entry points for advanced functionalities like pipelining and publish/subscribe. + + +**Related Classes/Methods**: + +- `redis.client.Redis` (112:670) +- `redis.client.Redis:from_url` (128:176) +- `redis.client.Redis:from_pool` (179:192) +- `redis.client.Redis:__init__` (199:389) +- `redis.client.Redis:get_retry` (405:406) +- `redis.client.Redis:set_retry` (408:410) +- `redis.client.Redis:pipeline` (439:449) +- `redis.client.Redis:transaction` (451:473) +- `redis.client.Redis:pubsub` (556:564) +- `redis.client.Redis:monitor` (566:567) +- `redis.client.Redis:client` (569:572) +- `redis.client.Redis:__exit__` (577:578) +- `redis.client.Redis:__del__` (580:584) +- `redis.client.Redis:_send_command_parse_response` (601:606) +- `redis.client.Redis:execute_command` (622:623) +- `redis.client.Redis:_execute_command` (625:644) + + +### Lock Management +The Lock Management component provides functionalities for acquiring, releasing, extending, and reacquiring distributed locks in Redis. It handles the underlying Redis commands and manages lock ownership and expiration, raising specific exceptions for lock-related errors. + + +**Related Classes/Methods**: + +- `redis.lock.Lock` (14:343) +- `redis.lock.Lock:__init__` (79:155) +- `redis.lock.Lock:__enter__` (167:173) +- `redis.lock.Lock:__exit__` (175:188) +- `redis.lock.Lock:acquire` (190:235) +- `redis.lock.Lock:release` (265:276) +- `redis.lock.Lock:do_acquire` (237:245) +- `redis.lock.Lock:do_release` (278:285) +- `redis.lock.Lock:extend` (287:302) +- `redis.lock.Lock:do_extend` (304:317) +- `redis.lock.Lock:reacquire` (319:330) +- `redis.lock.Lock:do_reacquire` (332:343) + + +### Pub/Sub Component +The Pub/Sub Component facilitates real-time messaging through Redis's publish/subscribe mechanism. It allows clients to subscribe to channels or patterns, receive messages, and manage the lifecycle of the pub/sub connection. + + +**Related Classes/Methods**: + +- `redis.client.PubSub` (743:1000) +- `redis.client.PubSub:__init__` (756:791) +- `redis.client.PubSub:__exit__` (796:797) +- `redis.client.PubSub:__del__` (799:806) +- `redis.client.PubSub:close` (823:824) +- `redis.client.PubSub:on_connect` (826:849) +- `redis.client.PubSub:execute_command` (856:880) +- `redis.client.PubSub:clean_health_check_responses` (882:898) +- `redis.client.PubSub:_execute` (910:921) +- `redis.client.PubSub:parse_response` (923:948) +- `redis.client.PubSub:psubscribe` (983:1000) +- `redis.client.PubSub:punsubscribe` (full file reference) +- `redis.client.PubSub:subscribe` (full file reference) +- `redis.client.PubSub:unsubscribe` (full file reference) +- `redis.client.PubSub:ssubscribe` (full file reference) +- `redis.client.PubSub:sunsubscribe` (full file reference) +- `redis.client.PubSub:listen` (full file reference) +- `redis.client.PubSub:get_message` (full file reference) +- `redis.client.PubSub:ping` (full file reference) +- `redis.client.PubSub:handle_message` (full file reference) +- `redis.client.PubSub:run_in_thread` (full file reference) + + +### Pipeline/Transaction Component +The Pipeline/Transaction Component enables efficient execution of multiple Redis commands by sending them in a single round trip (pipelining) and supports atomic transactions using MULTI/EXEC. It manages command queuing, response parsing, and error handling for batched operations. + + +**Related Classes/Methods**: + +- `redis.client.Pipeline` (1:1000) +- `redis.client.Pipeline:__exit__` (full file reference) +- `redis.client.Pipeline:__del__` (full file reference) +- `redis.client.Pipeline:close` (full file reference) +- `redis.client.Pipeline:multi` (full file reference) +- `redis.client.Pipeline:execute_command` (full file reference) +- `redis.client.Pipeline:_disconnect_reset_raise_on_watching` (full file reference) +- `redis.client.Pipeline:immediate_execute_command` (full file reference) +- `redis.client.Pipeline:_execute_transaction` (full file reference) +- `redis.client.Pipeline:_execute_pipeline` (full file reference) +- `redis.client.Pipeline:raise_first_error` (full file reference) +- `redis.client.Pipeline:annotate_exception` (full file reference) +- `redis.client.Pipeline:parse_response` (full file reference) +- `redis.client.Pipeline:load_scripts` (full file reference) +- `redis.client.Pipeline:_disconnect_raise_on_watching` (full file reference) +- `redis.client.Pipeline:execute` (full file reference) +- `redis.client.Pipeline:discard` (full file reference) +- `redis.client.Pipeline:watch` (full file reference) +- `redis.client.Pipeline:unwatch` (full file reference) + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/on_boarding.md b/.codeboarding/on_boarding.md new file mode 100644 index 0000000000..0c78802f5c --- /dev/null +++ b/.codeboarding/on_boarding.md @@ -0,0 +1,87 @@ +```mermaid +graph LR + Redis_Client_Core["Redis Client Core"] + Connection_Protocol_Management["Connection & Protocol Management"] + Command_Module_Execution["Command & Module Execution"] + High_Availability_Cluster_Management["High Availability & Cluster Management"] + Error_Handling["Error Handling"] + Redis_Client_Core -- "uses" --> Connection_Protocol_Management + Redis_Client_Core -- "executes" --> Command_Module_Execution + Redis_Client_Core -- "handles" --> Error_Handling + Connection_Protocol_Management -- "provides to" --> Redis_Client_Core + Connection_Protocol_Management -- "raises" --> Error_Handling + Command_Module_Execution -- "is executed by" --> Redis_Client_Core + Command_Module_Execution -- "uses" --> Connection_Protocol_Management + High_Availability_Cluster_Management -- "extends" --> Redis_Client_Core + High_Availability_Cluster_Management -- "uses" --> Connection_Protocol_Management + Error_Handling -- "is raised by" --> Redis_Client_Core + Error_Handling -- "is raised by" --> Connection_Protocol_Management + click Redis_Client_Core href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Redis Client Core.md" "Details" + click Connection_Protocol_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Connection & Protocol Management.md" "Details" + click Command_Module_Execution href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Command & Module Execution.md" "Details" + click High_Availability_Cluster_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/High Availability & Cluster Management.md" "Details" + click Error_Handling href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Error Handling.md" "Details" +``` +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + +## Component Details + +The `redis-py` library provides a Python interface to the Redis key-value store. Its main flow involves establishing connections to Redis servers, executing various Redis commands, handling responses, and managing advanced features like pipelining, transactions, and publish/subscribe. It also supports specialized deployments such as Redis Cluster and Redis Sentinel for high availability and scalability. The architecture is designed to be modular, separating concerns like connection management, command execution, and error handling. + +### Redis Client Core +The primary interface for interacting with Redis, encompassing basic command execution, pipelining, transactions, publish/subscribe, and distributed locking mechanisms. It orchestrates high-level operations. + + +**Related Classes/Methods**: + +- `redis.client.Redis` (112:670) +- `redis.client.Pipeline` (1:1000) +- `redis.client.PubSub` (743:1000) +- `redis.lock.Lock` (14:343) + + +### Connection & Protocol Management +Manages the lifecycle of connections to Redis servers, including pooling, health checks, and various connection types. It also handles the encoding of commands and parsing of responses, along with authentication and caching. + + +**Related Classes/Methods**: + +- `redis.connection.ConnectionPool` (1:1000) +- `redis._parsers.resp3._RESP3Parser` (15:131) +- `redis.auth.token_manager.TokenManager` (121:340) +- `redis.cache.DefaultCache` (142:224) +- `redis.ocsp.OCSPVerifier` (170:308) + + +### Command & Module Execution +Implements and executes the full range of standard Redis commands (e.g., key-value, list, set, hash, stream operations) and provides interfaces for interacting with various Redis Modules (e.g., JSON, Search, TimeSeries). + + +**Related Classes/Methods**: + +- `redis.commands.core.BasicKeyCommands` (1:1000) +- `redis.commands.json.JSON` (1:1000) + + +### High Availability & Cluster Management +Provides specialized client functionalities for interacting with Redis Cluster and Redis Sentinel setups. It handles node discovery, slot management, command routing in clusters, and master/replica discovery with failover in Sentinel environments. + + +**Related Classes/Methods**: + +- `redis.cluster.RedisCluster` (456:1000) +- `redis.sentinel.Sentinel` (198:410) + + +### Error Handling +Defines and manages custom exception classes for various Redis-related errors, providing a structured and specific way to handle different error scenarios that can occur during client-server interactions. + + +**Related Classes/Methods**: + +- `redis.exceptions.RedisError` (4:5) + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/CodeBoarding/Client Interface.md b/CodeBoarding/Client Interface.md deleted file mode 100644 index b5f9bae16d..0000000000 --- a/CodeBoarding/Client Interface.md +++ /dev/null @@ -1,52 +0,0 @@ -```mermaid -graph LR - Redis_Client["Redis Client"] - Connection_Pool["Connection Pool"] - Connection["Connection"] - Command_Parser["Command Parser"] - PubSub["PubSub"] - Pipeline["Pipeline"] - Monitor["Monitor"] - Redis_Client -- "creates" --> Connection_Pool - Redis_Client -- "uses" --> Command_Parser - Redis_Client -- "creates" --> PubSub - Redis_Client -- "creates" --> Pipeline - Redis_Client -- "creates" --> Monitor - Connection_Pool -- "uses" --> Connection - Pipeline -- "uses" --> Redis_Client - PubSub -- "uses" --> Redis_Client - Monitor -- "uses" --> Redis_Client -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The Redis client library provides a comprehensive interface for interacting with Redis servers, supporting both synchronous and asynchronous operations. It encompasses connection management, command execution, and response handling, offering features like PubSub, Pipelines, and Monitoring. The core components work together to provide a robust and efficient way to interact with Redis. - -### Redis Client -The Redis Client serves as the primary interface for interacting with a Redis server. It manages connections, executes commands, and handles responses. It supports both synchronous and asynchronous operations, allowing users to interact with Redis in a non-blocking manner. The client can be configured to connect to a single Redis instance or a Redis cluster. -- **Related Classes/Methods**: `redis.client.Redis` (112:670), `redis.asyncio.client.Redis` (109:715) - -### Connection Pool -The Connection Pool manages a pool of reusable connections to the Redis server. This improves performance by reducing the overhead of establishing new connections for each request. The connection pool handles connection creation, recycling, and error handling, ensuring efficient use of resources. -- **Related Classes/Methods**: `redis.connection.ConnectionPool` (1309:1654), `redis.asyncio.connection.ConnectionPool` (1031:1253) - -### Connection -The Connection class represents a single connection to the Redis server. It handles the low-level details of socket communication, including sending commands and receiving responses. It provides methods for reading and writing data to the socket, as well as handling connection errors. -- **Related Classes/Methods**: `redis.connection.Connection` (730:801), `redis.asyncio.connection.Connection` (723:777) - -### Command Parser -The Command Parser is responsible for parsing the responses received from the Redis server. It converts the raw byte strings into Python data types, such as strings, integers, lists, and dictionaries. The parser handles different response formats and error conditions, ensuring that the data is correctly interpreted. -- **Related Classes/Methods**: `redis.client.Redis.parse_response` (646:667), `redis.asyncio.client.Redis.parse_response` (689:715) - -### PubSub -The PubSub class provides functionality for publishing messages to channels and subscribing to channels to receive messages. It enables real-time communication between clients using the publish-subscribe pattern. It supports pattern subscriptions and message filtering. -- **Related Classes/Methods**: `redis.client.PubSub` (743:1241), `redis.asyncio.client.PubSub` (803:1231) - -### Pipeline -The Pipeline class allows batching multiple commands into a single request, reducing network overhead and improving performance. It supports transactions, allowing a group of commands to be executed atomically. It also supports optimistic locking using WATCH and UNWATCH commands. -- **Related Classes/Methods**: `redis.client.Pipeline` (1283:1635), `redis.asyncio.client.Pipeline` (1251:1618) - -### Monitor -The Monitor class allows you to listen to all requests received by the Redis server in real time. It's useful for debugging and monitoring Redis activity. It provides a stream of commands and their arguments as they are processed by the server. -- **Related Classes/Methods**: `redis.client.Monitor` (676:740), `redis.asyncio.client.Monitor` (730:800) \ No newline at end of file diff --git a/CodeBoarding/Cluster Support.md b/CodeBoarding/Cluster Support.md deleted file mode 100644 index c58179e45e..0000000000 --- a/CodeBoarding/Cluster Support.md +++ /dev/null @@ -1,53 +0,0 @@ -```mermaid -graph LR - RedisCluster["RedisCluster"] - NodesManager["NodesManager"] - ClusterPubSub["ClusterPubSub"] - ClusterPipeline["ClusterPipeline"] - PipelineStrategy["PipelineStrategy"] - TransactionStrategy["TransactionStrategy"] - ClusterMultiKeyCommands["ClusterMultiKeyCommands"] - RedisCluster -- "manages" --> NodesManager - RedisCluster -- "uses" --> ClusterPubSub - RedisCluster -- "uses" --> ClusterPipeline - RedisCluster -- "determines node for" --> ClusterMultiKeyCommands - NodesManager -- "provides node information to" --> RedisCluster - ClusterPubSub -- "executes commands" --> RedisCluster - ClusterPipeline -- "executes commands" --> RedisCluster - PipelineStrategy -- "implements" --> AbstractStrategy - TransactionStrategy -- "implements" --> AbstractStrategy - ClusterMultiKeyCommands -- "uses" --> RedisCluster -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The Cluster Support component in redis-py provides the necessary tools for interacting with Redis clusters. It abstracts the complexities of cluster management, such as node discovery, slot assignment, and command routing, offering a unified interface for both synchronous and asynchronous operations. The core of this component lies in the `RedisCluster` class, which acts as the primary client. It leverages the `NodesManager` to maintain an updated view of the cluster topology and uses strategies like `PipelineStrategy` and `TransactionStrategy` to optimize command execution. The component also includes specialized classes for pub/sub (`ClusterPubSub`) and multi-key commands (`ClusterMultiKeyCommands`), ensuring comprehensive cluster functionality. - -### RedisCluster -The RedisCluster class serves as the primary client interface for interacting with a Redis cluster. It handles connection management, command execution, and slot assignment to nodes in the cluster. It uses the NodesManager to maintain an up-to-date view of the cluster topology and routes commands to the appropriate nodes based on the key's slot. It supports both synchronous and asynchronous operations. -- **Related Classes/Methods**: `redis.cluster.RedisCluster` (456:1360), `redis.asyncio.cluster.RedisCluster` (99:989) - -### NodesManager -The NodesManager class is responsible for managing the cluster's node topology. It maintains a list of all nodes in the cluster, their roles (primary or replica), and the slot ranges they serve. It dynamically updates the topology when nodes are added or removed, ensuring that the RedisCluster always has an accurate view of the cluster's structure. -- **Related Classes/Methods**: `redis.cluster.NodesManager` (1443:1863), `redis.asyncio.cluster.NodesManager` (1211:1518) - -### ClusterPubSub -The ClusterPubSub class provides a client interface for interacting with Redis cluster's pub/sub functionality. It handles subscribing to channels and publishing messages to channels across the cluster, ensuring that messages are correctly routed to the appropriate nodes. -- **Related Classes/Methods**: `redis.cluster.ClusterPubSub` (1866:2107) - -### ClusterPipeline -The ClusterPipeline class enables the execution of a batch of commands in a pipeline on a Redis cluster. It efficiently routes commands to the correct nodes and executes them in parallel, reducing network overhead and improving performance. It supports both synchronous and asynchronous operations. -- **Related Classes/Methods**: `redis.cluster.ClusterPipeline` (2110:2299), `redis.asyncio.cluster.ClusterPipeline` (1521:1680) - -### PipelineStrategy -The PipelineStrategy class implements the AbstractStrategy interface for executing commands in a Redis cluster pipeline. It handles routing commands to the correct nodes and executing them in parallel, optimizing performance for pipelined operations. -- **Related Classes/Methods**: `redis.cluster.PipelineStrategy` (2671:3018), `redis.asyncio.cluster.PipelineStrategy` (1872:2039) - -### TransactionStrategy -The TransactionStrategy class implements the AbstractStrategy interface for executing commands in a Redis cluster transaction. It ensures that all commands within the transaction are executed atomically, providing data consistency and reliability. -- **Related Classes/Methods**: `redis.cluster.TransactionStrategy` (3021:3352), `redis.asyncio.cluster.TransactionStrategy` (2042:2398) - -### ClusterMultiKeyCommands -The ClusterMultiKeyCommands class provides methods for executing multi-key commands on a Redis cluster. It handles partitioning keys by slot and routing commands to the correct nodes, ensuring that multi-key operations are performed efficiently across the cluster. -- **Related Classes/Methods**: `redis.commands.cluster.ClusterMultiKeyCommands` (99:260), `redis.commands.cluster.AsyncClusterMultiKeyCommands` (263:339) \ No newline at end of file diff --git a/CodeBoarding/Command Abstraction.md b/CodeBoarding/Command Abstraction.md deleted file mode 100644 index 26fbca8edf..0000000000 --- a/CodeBoarding/Command Abstraction.md +++ /dev/null @@ -1,66 +0,0 @@ -```mermaid -graph LR - Command_Interface["Command Interface"] - Basic_Key_Commands["Basic Key Commands"] - Hash_Commands["Hash Commands"] - List_Commands["List Commands"] - Set_Commands["Set Commands"] - Sorted_Set_Commands["Sorted Set Commands"] - Stream_Commands["Stream Commands"] - PubSub_Commands["PubSub Commands"] - Script_Commands["Script Commands"] - Basic_Key_Commands -- "Implements" --> Command_Interface - Hash_Commands -- "Implements" --> Command_Interface - List_Commands -- "Implements" --> Command_Interface - Set_Commands -- "Implements" --> Command_Interface - Sorted_Set_Commands -- "Implements" --> Command_Interface - Stream_Commands -- "Implements" --> Command_Interface - PubSub_Commands -- "Implements" --> Command_Interface - Script_Commands -- "Implements" --> Command_Interface - List_Commands -- "Uses" --> Basic_Key_Commands - Set_Commands -- "Uses" --> Basic_Key_Commands - Stream_Commands -- "Uses" --> Basic_Key_Commands - Sorted_Set_Commands -- "Uses" --> Basic_Key_Commands - Hash_Commands -- "Uses" --> Basic_Key_Commands -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The Command Abstraction component provides a unified interface for interacting with Redis, abstracting away the complexities of command encoding, decoding, and execution. It encompasses various command categories, including keys, hashes, lists, sets, sorted sets, streams, pubsub, scripts, geo, modules, and functions, offering both synchronous and asynchronous execution modes. This abstraction ensures a consistent API for developers, regardless of the underlying Redis connection type, and simplifies the process of interacting with the Redis server. - -### Command Interface -Defines the base interface for all Redis commands, providing a consistent way to execute commands and handle responses. It serves as a blueprint for concrete command implementations. -- **Related Classes/Methods**: `redis.commands.core.CommandsInterface` (20:100) - -### Basic Key Commands -Implements basic key-related commands such as GET, SET, EXISTS, and DELETE. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding key values. -- **Related Classes/Methods**: `redis.commands.core.BasicKeyCommands` (1557:2510) - -### Hash Commands -Implements commands for interacting with Redis hashes, including setting, getting, and deleting fields. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding hash field values. -- **Related Classes/Methods**: `redis.commands.core.HashCommands` (4921:5598) - -### List Commands -Implements commands for interacting with Redis lists, including pushing, popping, and trimming elements. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding list element values. -- **Related Classes/Methods**: `redis.commands.core.ListCommands` (2533:2947) - -### Set Commands -Implements commands for interacting with Redis sets, including adding, removing, and checking membership of elements. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding set element values. -- **Related Classes/Methods**: `redis.commands.core.SetCommands` (3287:3462) - -### Sorted Set Commands -Implements commands for interacting with Redis sorted sets, including adding, removing, and retrieving elements with scores. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding sorted set element values. -- **Related Classes/Methods**: `redis.commands.core.SortedSetCommands` (4077:4870) - -### Stream Commands -Implements commands for interacting with Redis streams, including adding messages, reading messages, and creating consumer groups. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding stream message values. -- **Related Classes/Methods**: `redis.commands.core.StreamCommands` (3468:4071) - -### PubSub Commands -Implements commands for interacting with Redis's Pub/Sub functionality, including publishing messages and subscribing to channels. It interacts with the Redis client to execute these commands and provides methods for encoding and decoding pubsub messages. -- **Related Classes/Methods**: `redis.commands.core.PubSubCommands` (5720:5784) - -### Script Commands -Enables the execution of Lua scripts on the Redis server. It includes functionalities for evaluating, loading, and managing scripts, providing a way to extend Redis's capabilities with custom logic. -- **Related Classes/Methods**: `redis.commands.core.ScriptCommands` (5790:5928) \ No newline at end of file diff --git a/CodeBoarding/Connection Management.md b/CodeBoarding/Connection Management.md deleted file mode 100644 index abae81ce86..0000000000 --- a/CodeBoarding/Connection Management.md +++ /dev/null @@ -1,66 +0,0 @@ -```mermaid -graph LR - AbstractConnection["AbstractConnection"] - Connection["Connection"] - SSLConnection["SSLConnection"] - UnixDomainSocketConnection["UnixDomainSocketConnection"] - ConnectionPool["ConnectionPool"] - BlockingConnectionPool["BlockingConnectionPool"] - AbstractConnection_asyncio_["AbstractConnection (asyncio)"] - Connection_asyncio_["Connection (asyncio)"] - ConnectionPool_asyncio_["ConnectionPool (asyncio)"] - SentinelConnectionPool["SentinelConnectionPool"] - AbstractConnection -- "is a base class for" --> Connection - AbstractConnection -- "is a base class for" --> SSLConnection - AbstractConnection -- "is a base class for" --> UnixDomainSocketConnection - ConnectionPool -- "manages" --> Connection - ConnectionPool -- "extends" --> BlockingConnectionPool - AbstractConnection_asyncio_ -- "is a base class for" --> Connection_asyncio_ - ConnectionPool_asyncio_ -- "manages" --> Connection_asyncio_ - SentinelConnectionPool -- "manages" --> Connection -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The Connection Management component in `redis` library is responsible for managing connections to Redis servers. It provides connection pooling, different connection types (TCP, SSL, Unix sockets), and connection management functionalities like authentication and health checks. The core of this component revolves around the `AbstractConnection` and its implementations, along with `ConnectionPool` which manages the connections. Sentinel-related classes handle connections in a Sentinel-managed Redis setup. - -### AbstractConnection -AbstractConnection defines the base connection interface and implements common connection logic. It handles connection establishment, health checks, command sending, and response reading. It serves as a parent class for concrete connection implementations, providing a foundation for different connection types. -- **Related Classes/Methods**: `redis.connection.AbstractConnection:__init__` (228:322), `redis.connection.AbstractConnection:__repr__` (324:326), `redis.connection.AbstractConnection:__del__` (332:336), `redis.connection.AbstractConnection:_construct_command_packer` (338:344), `redis.connection.AbstractConnection:connect` (377:379), `redis.connection.AbstractConnection:connect_check_health` (381:413), `redis.connection.AbstractConnection:_error_message` (423:424), `redis.connection.AbstractConnection:on_connect` (426:427), `redis.connection.AbstractConnection:on_connect_check_health` (429:538), `redis.connection.AbstractConnection:_send_ping` (560:564), `redis.connection.AbstractConnection:_ping_failed` (566:568), `redis.connection.AbstractConnection:check_health` (570:573), `redis.connection.AbstractConnection:send_packed_command` (575:604), `redis.connection.AbstractConnection:send_command` (606:611), `redis.connection.AbstractConnection:can_read` (613:625), `redis.connection.AbstractConnection:read_response` (627:669), `redis.connection.AbstractConnection:re_auth` (719:727) - -### Connection -Connection is a concrete implementation of AbstractConnection for standard TCP connections. It inherits connection management and command execution logic from AbstractConnection, providing a basic TCP socket connection to Redis. -- **Related Classes/Methods**: `redis.connection.Connection:__init__` (733:747) - -### SSLConnection -SSLConnection extends Connection to provide SSL/TLS encryption for secure communication with Redis. It handles SSL context creation and socket wrapping, ensuring secure data transmission. -- **Related Classes/Methods**: `redis.connection.SSLConnection:__init__` (1024:1096), `redis.connection.SSLConnection:_connect` (1098:1107), `redis.connection.SSLConnection:_wrap_socket_with_ssl` (1109:1186) - -### UnixDomainSocketConnection -UnixDomainSocketConnection extends Connection for communication over Unix domain sockets. It adapts the connection process to use socket files instead of TCP addresses, enabling local communication with Redis. -- **Related Classes/Methods**: `redis.connection.UnixDomainSocketConnection:__init__` (1192:1195) - -### ConnectionPool -ConnectionPool manages a pool of connections to a Redis server. It handles connection creation, retrieval, and release, optimizing connection reuse and reducing connection overhead. It also manages the encoder to use for the connections, improving performance and resource utilization. -- **Related Classes/Methods**: `redis.connection.ConnectionPool:from_url` (1324:1370), `redis.connection.ConnectionPool:__init__` (1372:1433), `redis.connection.ConnectionPool:_checkpid` (1467:1512), `redis.connection.ConnectionPool:get_connection` (1519:1551), `redis.connection.ConnectionPool:get_encoder` (1553:1560), `redis.connection.ConnectionPool:make_connection` (1562:1573), `redis.connection.ConnectionPool:release` (1575:1597), `redis.connection.ConnectionPool:disconnect` (1602:1620), `redis.connection.ConnectionPool:close` (1622:1624), `redis.connection.ConnectionPool:re_auth_callback` (1633:1646) - -### BlockingConnectionPool -BlockingConnectionPool extends ConnectionPool to provide blocking behavior when retrieving connections from the pool. It waits for a connection to become available if the pool is exhausted, ensuring that a connection is eventually obtained. -- **Related Classes/Methods**: `redis.connection.BlockingConnectionPool:__init__` (1691:1705), `redis.connection.BlockingConnectionPool:make_connection` (1731:1740), `redis.connection.BlockingConnectionPool:get_connection` (1747:1797), `redis.connection.BlockingConnectionPool:release` (1799:1818), `redis.connection.BlockingConnectionPool:disconnect` (1820:1824) - -### AbstractConnection (asyncio) -AbstractConnection (asyncio) defines the base connection interface and implements common connection logic for asynchronous connections. It handles connection establishment, health checks, command sending, and response reading in an asynchronous manner. -- **Related Classes/Methods**: `redis.asyncio.connection.AbstractConnection:__init__` (138:225), `redis.asyncio.connection.AbstractConnection:__del__` (227:241), `redis.asyncio.connection.AbstractConnection:__repr__` (251:253), `redis.asyncio.connection.AbstractConnection:connect` (294:296), `redis.asyncio.connection.AbstractConnection:connect_check_health` (298:338), `redis.asyncio.connection.AbstractConnection:_error_message` (348:349), `redis.asyncio.connection.AbstractConnection:on_connect` (354:356), `redis.asyncio.connection.AbstractConnection:on_connect_check_health` (358:464), `redis.asyncio.connection.AbstractConnection:disconnect` (466:487), `redis.asyncio.connection.AbstractConnection:_send_ping` (489:493), `redis.asyncio.connection.AbstractConnection:_ping_failed` (495:497), `redis.asyncio.connection.AbstractConnection:send_packed_command` (511:550), `redis.asyncio.connection.AbstractConnection:send_command` (552:556), `redis.asyncio.connection.AbstractConnection:can_read_destructive` (558:565), `redis.asyncio.connection.AbstractConnection:read_response` (567:623), `redis.asyncio.connection.AbstractConnection:pack_commands` (671:699), `redis.asyncio.connection.AbstractConnection:process_invalidation_messages` (705:707), `redis.asyncio.connection.AbstractConnection:re_auth` (712:720) - -### Connection (asyncio) -Connection (asyncio) is a concrete implementation of AbstractConnection for standard TCP connections in an asynchronous context. It inherits connection management and command execution logic from AbstractConnection, providing an asynchronous TCP socket connection to Redis. -- **Related Classes/Methods**: `redis.asyncio.connection.Connection:__init__` (726:741), `redis.asyncio.connection.Connection:_connect` (752:774) - -### ConnectionPool (asyncio) -ConnectionPool (asyncio) manages a pool of asynchronous connections to a Redis server. It handles connection creation, retrieval, and release, optimizing connection reuse and reducing connection overhead in asynchronous operations. -- **Related Classes/Methods**: `redis.asyncio.connection.ConnectionPool:from_url` (1046:1090), `redis.asyncio.connection.ConnectionPool:__init__` (1092:1112), `redis.asyncio.connection.ConnectionPool:get_connection` (1138:1148), `redis.asyncio.connection.ConnectionPool:get_available_connection` (1150:1159), `redis.asyncio.connection.ConnectionPool:make_connection` (1170:1172), `redis.asyncio.connection.ConnectionPool:ensure_connection` (1174:1188), `redis.asyncio.connection.ConnectionPool:release` (1190:1198), `redis.asyncio.connection.ConnectionPool:aclose` (1222:1224), `redis.asyncio.connection.ConnectionPool:re_auth_callback` (1232:1245) - -### SentinelConnectionPool -SentinelConnectionPool manages a pool of connections to Redis servers monitored by Sentinel. It handles connection discovery, failover, and role management based on Sentinel's information, ensuring high availability and fault tolerance. -- **Related Classes/Methods**: `redis.sentinel.SentinelConnectionPool:__init__` (145:166), `redis.sentinel.SentinelConnectionPool:reset` (175:177), `redis.sentinel.SentinelConnectionPool:owns_connection` (183:188), `redis.sentinel.SentinelConnectionPool:get_master_address` (190:191), `redis.sentinel.SentinelConnectionPool:rotate_slaves` (193:195) \ No newline at end of file diff --git a/CodeBoarding/Data Handling.md b/CodeBoarding/Data Handling.md deleted file mode 100644 index bb7c73e731..0000000000 --- a/CodeBoarding/Data Handling.md +++ /dev/null @@ -1,54 +0,0 @@ -```mermaid -graph LR - Encoder["Encoder"] - CommandsParser["CommandsParser"] - BaseParser["BaseParser"] - _RESP2Parser["_RESP2Parser"] - _RESP3Parser["_RESP3Parser"] - _HiredisParser["_HiredisParser"] - SocketBuffer["SocketBuffer"] - Helpers["Helpers"] - Encoder -- "encodes data with" --> CommandsParser - SocketBuffer -- "reads from" --> _HiredisParser - SocketBuffer -- "reads from" --> BaseParser - _HiredisParser -- "parses responses with" --> Helpers - _RESP3Parser -- "parses responses with" --> Helpers - _RESP2Parser -- "parses responses with" --> Helpers -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The Data Handling component is responsible for managing the serialization of commands and parsing of responses between the client and the Redis server. It encompasses encoding outgoing commands into the Redis Serialization Protocol (RESP) format and decoding incoming responses back into Python data types. This involves handling different RESP versions (RESP2, RESP3) and utilizing helper functions for parsing specific data structures. The component ensures data integrity and compatibility during communication with the Redis server. - -### Encoder -The Encoder class handles the serialization of Python data into the Redis Serialization Protocol (RESP) format. It provides methods for encoding various data types, ensuring they are properly formatted for transmission to the Redis server. It interacts with the CommandsParser to encode commands before sending them to Redis. -- **Related Classes/Methods**: `redis._parsers.encoders.Encoder:encode` (14:35) - -### CommandsParser -The CommandsParser class is responsible for parsing commands and extracting keys from them. It is initialized with command flags and provides methods for retrieving keys based on the command being parsed. It interacts with the Encoder to prepare commands for encoding and transmission. -- **Related Classes/Methods**: `redis._parsers.commands.AbstractCommandsParser:_get_pubsub_keys` (11:38), `redis._parsers.commands.AbstractCommandsParser:parse_subcommand` (40:53), `redis._parsers.commands.CommandsParser:__init__` (65:67), `redis._parsers.commands.CommandsParser:get_keys` (82:145), `redis._parsers.asyncio.commands.AsyncCommandsParser:get_keys` (full file reference) - -### BaseParser -The BaseParser class serves as the base class for all parsers (RESP2, RESP3, Hiredis). It provides common functionality and defines the interface for parsing responses. It includes methods for handling errors and managing the connection. It interacts with the SocketBuffer to read data and with the Helpers module to parse specific data types. -- **Related Classes/Methods**: `redis._parsers.base.BaseParser:parse_error` (90:99), `redis._parsers.base._RESPBase:__del__` (117:121), `redis._parsers.base._RESPBase:on_connect` (123:129), `redis._parsers.base._RESPBase:can_read` (139:140), `redis._parsers.base._AsyncRESPBase:__init__` (218:223), `redis._parsers.base._AsyncRESPBase:on_connect` (229:236), `redis._parsers.base._AsyncRESPBase:can_read_destructive` (242:251), `redis._parsers.base._AsyncRESPBase:_read` (253:271), `redis._parsers.base._AsyncRESPBase:_readline` (273:289) - -### _RESP2Parser -The _RESP2Parser class is responsible for parsing RESP2 responses. It provides methods for reading and parsing responses according to the RESP2 protocol. It handles the parsing of data structures defined in RESP2. It inherits from BaseParser and uses the Helpers module for parsing specific data types. -- **Related Classes/Methods**: `redis._parsers.resp2._RESP2Parser:read_response` (12:22), `redis._parsers.resp2._RESP2Parser:_read_response` (24:68), `redis._parsers.resp2._AsyncRESP2Parser:read_response` (74:85), `redis._parsers.resp2._AsyncRESP2Parser:_read_response` (87:132) - -### _RESP3Parser -The _RESP3Parser class is responsible for parsing RESP3 responses. It provides methods for reading and parsing responses according to the RESP3 protocol. It handles the complexities of the RESP3 protocol, including different data types and structures. It inherits from BaseParser and uses the Helpers module for parsing specific data types. -- **Related Classes/Methods**: `redis._parsers.resp3._RESP3Parser:__init__` (18:21), `redis._parsers.resp3._RESP3Parser:read_response` (28:40), `redis._parsers.resp3._RESP3Parser:_read_response` (42:131), `redis._parsers.resp3._AsyncRESP3Parser:__init__` (135:138), `redis._parsers.resp3._AsyncRESP3Parser:read_response` (145:158), `redis._parsers.resp3._AsyncRESP3Parser:_read_response` (160:257) - -### _HiredisParser -The _HiredisParser class is a parser that uses the hiredis library for parsing RESP (Redis Serialization Protocol) responses. It provides methods for reading data from the socket and parsing the response. It leverages the speed and efficiency of the hiredis library for faster parsing. It inherits from BaseParser and uses the Helpers module for parsing specific data types. -- **Related Classes/Methods**: `redis._parsers.hiredis._HiredisParser:__init__` (44:51), `redis._parsers.hiredis._HiredisParser:__del__` (53:57), `redis._parsers.hiredis._HiredisParser:can_read` (92:100), `redis._parsers.hiredis._HiredisParser:read_from_socket` (102:130), `redis._parsers.hiredis._HiredisParser:read_response` (132:184), `redis._parsers.hiredis._AsyncHiredisParser:__init__` (192:199), `redis._parsers.hiredis._AsyncHiredisParser:can_read_destructive` (233:242), `redis._parsers.hiredis._AsyncHiredisParser:read_from_socket` (244:251), `redis._parsers.hiredis._AsyncHiredisParser:read_response` (253:295) - -### SocketBuffer -The SocketBuffer class manages reading data from a socket. It provides methods for reading lines, reading a specific number of bytes, checking if data is available, and purging the buffer. It acts as an intermediary between the raw socket and the parsers, providing a buffered interface for reading data. It is used by the parsers (RESP2, RESP3, Hiredis) to read data from the socket. -- **Related Classes/Methods**: `redis._parsers.socket.SocketBuffer:_read_from_socket` (47:92), `redis._parsers.socket.SocketBuffer:can_read` (94:97), `redis._parsers.socket.SocketBuffer:read` (99:108), `redis._parsers.socket.SocketBuffer:readline` (110:118), `redis._parsers.socket.SocketBuffer:purge` (132:149) - -### Helpers -The Helpers module contains a collection of helper functions for parsing various Redis responses. These functions are used to extract specific data from the responses and convert them into Python data types. They provide specialized parsing logic for different Redis commands and data structures. It is used by the parsers (RESP2, RESP3, Hiredis) to parse specific data types. -- **Related Classes/Methods**: `redis._parsers.helpers:parse_debug_object` (17:32), `redis._parsers.helpers:parse_info` (35:83), `redis._parsers.helpers:parse_memory_stats` (86:94), `redis._parsers.helpers:parse_sentinel_state` (124:137), `redis._parsers.helpers:parse_sentinel_master` (140:141), `redis._parsers.helpers:parse_sentinel_state_resp3` (144:154), `redis._parsers.helpers:parse_sentinel_masters` (157:162), `redis._parsers.helpers:parse_sentinel_masters_resp3` (165:166), `redis._parsers.helpers:parse_sentinel_slaves_and_sentinels` (169:170), `redis._parsers.helpers:parse_sentinel_slaves_and_sentinels_resp3` (173:174), `redis._parsers.helpers:parse_stream_list` (238:247), `redis._parsers.helpers:pairs_to_dict_with_str_keys` (250:251), `redis._parsers.helpers:parse_xclaim` (258:261), `redis._parsers.helpers:parse_xautoclaim` (264:268), `redis._parsers.helpers:parse_xinfo_stream` (271:299), `redis._parsers.helpers:parse_xread` (302:305), `redis._parsers.helpers:parse_xread_resp3` (308:311), `redis._parsers.helpers:parse_xpending` (314:323), `redis._parsers.helpers:bool_ok` (337:338), `redis._parsers.helpers:parse_client_list` (349:354), `redis._parsers.helpers:parse_config_get` (357:359), `redis._parsers.helpers:parse_hscan` (367:374), `redis._parsers.helpers:parse_slowlog_get` (389:415), `redis._parsers.helpers:parse_stralgo` (418:444), `redis._parsers.helpers:parse_cluster_info` (447:449), `redis._parsers.helpers:_parse_node_line` (452:472), `redis._parsers.helpers:parse_cluster_nodes` (495:502), `redis._parsers.helpers:parse_command` (541:557), `redis._parsers.helpers:parse_command_resp3` (560:578), `redis._parsers.helpers:parse_client_kill` (585:588), `redis._parsers.helpers:parse_acl_getuser` (591:631), `redis._parsers.helpers:parse_acl_log` (634:649), `redis._parsers.helpers:parse_client_info` (652:680), `redis._parsers.helpers:parse_config_get` (357:359), `redis._parsers.helpers:parse_client_info` (652:680), `redis._parsers.helpers:parse_set_result` (683:694) \ No newline at end of file diff --git a/CodeBoarding/PubSub Management.md b/CodeBoarding/PubSub Management.md deleted file mode 100644 index 321174a318..0000000000 --- a/CodeBoarding/PubSub Management.md +++ /dev/null @@ -1,43 +0,0 @@ -```mermaid -graph LR - PubSub_redis_client_["PubSub (redis.client)"] - PubSub_redis_asyncio_client_["PubSub (redis.asyncio.client)"] - Redis_redis_client_["Redis (redis.client)"] - Redis_redis_asyncio_client_["Redis (redis.asyncio.client)"] - PubSubCommands["PubSubCommands"] - ClusterPubSub_redis_cluster_["ClusterPubSub (redis.cluster)"] - Redis_redis_client_ -- "creates" --> PubSub_redis_client_ - Redis_redis_asyncio_client_ -- "creates" --> PubSub_redis_asyncio_client_ - PubSub_redis_client_ -- "uses" --> PubSubCommands - PubSub_redis_asyncio_client_ -- "uses" --> PubSubCommands - ClusterPubSub_redis_cluster_ -- "creates" --> PubSub_redis_client_ -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The PubSub Management component in Redis provides publish/subscribe functionalities for real-time messaging. Clients can subscribe to channels and receive messages published to those channels. It supports both synchronous and asynchronous pub/sub operations, managing subscriptions and efficiently distributing messages to subscribers. The core components include the synchronous and asynchronous PubSub classes, the Redis client classes that provide access to PubSub instances, the ClusterPubSub class for cluster environments, and the PubSubCommands class that defines the core pubsub commands. - -### PubSub (redis.client) -The PubSub class in redis.client provides a synchronous interface for subscribing to channels and listening for messages. It manages connection details, command execution, message parsing, and thread management for asynchronous message handling within a thread. -- **Related Classes/Methods**: `redis.client.PubSub` (743:1241), `redis.client.PubSub.__init__` (756:791), `redis.client.PubSub.__exit__` (796:797), `redis.client.PubSub.__del__` (799:806), `redis.client.PubSub:close` (823:824), `redis.client.PubSub:on_connect` (826:849), `redis.client.PubSub:execute_command` (856:880), `redis.client.PubSub:_execute` (910:921), `redis.client.PubSub:parse_response` (923:948), `redis.client.PubSub:psubscribe` (983:1007), `redis.client.PubSub:punsubscribe` (1009:1020), `redis.client.PubSub:subscribe` (1022:1046), `redis.client.PubSub:unsubscribe` (1048:1059), `redis.client.PubSub:ssubscribe` (1061:1085), `redis.client.PubSub:sunsubscribe` (1087:1098), `redis.client.PubSub:listen` (1100:1105), `redis.client.PubSub:get_message` (1107:1134), `redis.client.PubSub:ping` (1138:1143), `redis.client.PubSub:handle_message` (1145:1217), `redis.client.PubSub:run_in_thread` (1219:1241), `redis.client.PubSub:clean_health_check_responses` (882:898) - -### PubSub (redis.asyncio.client) -The PubSub class in redis.asyncio.client provides an asynchronous interface for subscribing to channels and listening for messages. It manages connections, executes commands, parses responses, and handles asynchronous message processing using asyncio. -- **Related Classes/Methods**: `redis.asyncio.client.PubSub` (803:1231), `redis.asyncio.client.PubSub.__init__` (816:855), `redis.asyncio.client.PubSub.__aexit__` (860:861), `redis.asyncio.client.PubSub:close` (885:887), `redis.asyncio.client.PubSub:reset` (890:892), `redis.asyncio.client.PubSub:on_connect` (894:910), `redis.asyncio.client.PubSub:execute_command` (917:927), `redis.asyncio.client.PubSub:connect` (929:947), `redis.asyncio.client.PubSub:_execute` (956:967), `redis.asyncio.client.PubSub:parse_response` (969:995), `redis.asyncio.client.PubSub:psubscribe` (1023:1042), `redis.asyncio.client.PubSub:punsubscribe` (1044:1057), `redis.asyncio.client.PubSub:subscribe` (1059:1078), `redis.asyncio.client.PubSub:unsubscribe` (1080:1092), `redis.asyncio.client.PubSub:listen` (1094:1099), `redis.asyncio.client.PubSub:get_message` (1101:1114), `redis.asyncio.client.PubSub:ping` (1116:1121), `redis.asyncio.client.PubSub:handle_message` (1123:1187), `redis.asyncio.client.PubSub:run` (1189:1231) - -### Redis (redis.client) -The Redis class in redis.client provides the base synchronous Redis client. It exposes the `pubsub` method, which returns a PubSub instance associated with that client, allowing clients to subscribe to channels and receive messages. -- **Related Classes/Methods**: `redis.client.Redis:pubsub` (556:564) - -### Redis (redis.asyncio.client) -The Redis class in redis.asyncio.client provides the base asynchronous Redis client. It exposes the `pubsub` method, which returns a PubSub instance associated with that client, enabling asynchronous subscription to channels and message reception. -- **Related Classes/Methods**: `redis.asyncio.client.Redis:pubsub` (578:586) - -### PubSubCommands -The PubSubCommands class in redis.commands.core provides the core pubsub commands such as publish, spublish, pubsub_channels, pubsub_numpat, pubsub_numsub, and pubsub_shardnumsub. These commands are used to publish messages to channels, retrieve information about active channels, and manage subscriptions. -- **Related Classes/Methods**: `redis.commands.core.PubSubCommands:publish` (5726:5733), `redis.commands.core.PubSubCommands:spublish` (5735:5742), `redis.commands.core.PubSubCommands:pubsub_channels` (5744:5750), `redis.commands.core.PubSubCommands:pubsub_shardchannels` (5752:5758), `redis.commands.core.PubSubCommands:pubsub_numpat` (5760:5766), `redis.commands.core.PubSubCommands:pubsub_numsub` (5768:5775), `redis.commands.core.PubSubCommands:pubsub_shardnumsub` (5777:5784) - -### ClusterPubSub (redis.cluster) -The ClusterPubSub class in redis.cluster provides an interface for pubsub operations in a Redis Cluster environment. It handles sharded messages and manages connections to multiple nodes to ensure messages are correctly distributed across the cluster. -- **Related Classes/Methods**: `redis.cluster.RedisCluster:pubsub` (834:839), `redis.cluster.ClusterPubSub` (1866:2107), `redis.cluster.ClusterPubSub.__init__` (1875:1918), `redis.cluster.ClusterPubSub:set_pubsub_node` (1920:1951), `redis.cluster.ClusterPubSub:_raise_on_invalid_node` (1959:1967), `redis.cluster.ClusterPubSub:execute_command` (1969:2009), `redis.cluster.ClusterPubSub:get_sharded_message` (2033:2057), `redis.cluster.ClusterPubSub:ssubscribe` (2059:2077), `redis.cluster.ClusterPubSub:sunsubscribe` (2079:2091) \ No newline at end of file diff --git a/CodeBoarding/on_boarding.md b/CodeBoarding/on_boarding.md deleted file mode 100644 index dbc624c0d4..0000000000 --- a/CodeBoarding/on_boarding.md +++ /dev/null @@ -1,50 +0,0 @@ -```mermaid -graph LR - Client_Interface["Client Interface"] - Command_Abstraction["Command Abstraction"] - Connection_Management["Connection Management"] - Cluster_Support["Cluster Support"] - PubSub_Management["PubSub Management"] - Data_Handling["Data Handling"] - Client_Interface -- "Uses" --> Connection_Management - Client_Interface -- "Uses" --> Command_Abstraction - Client_Interface -- "Uses" --> Data_Handling - Cluster_Support -- "Uses" --> Connection_Management - Command_Abstraction -- "Uses" --> Client_Interface - PubSub_Management -- "Uses" --> Client_Interface - click Client_Interface href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Client Interface.md" "Details" - click Command_Abstraction href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Command Abstraction.md" "Details" - click Connection_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Connection Management.md" "Details" - click Cluster_Support href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Cluster Support.md" "Details" - click PubSub_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/PubSub Management.md" "Details" - click Data_Handling href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Data Handling.md" "Details" -``` -[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20codeboarding@gmail.com-lightgrey?style=flat-square)](mailto:codeboarding@gmail.com) - -## Component Details - -The redis-py library provides a Python interface for interacting with Redis, a popular in-memory data structure store. The library abstracts the complexities of the Redis protocol, offering a user-friendly API for performing various operations, including data manipulation, pub/sub, and cluster management. It supports both synchronous and asynchronous operations, catering to a wide range of application requirements. The library is designed to be efficient and reliable, providing robust connection management and error handling. - -### Client Interface -The Client Interface serves as the primary entry point for interacting with Redis. It encapsulates connection management, command execution, and response handling. Supporting both synchronous and asynchronous operations, it can be configured to connect to a single Redis instance or a Redis cluster. This component handles the core functionality of sending commands to the Redis server and receiving responses, providing a high-level API for users. -- **Related Classes/Methods**: `redis.client.Redis` (112:670), `redis.asyncio.client.Redis` (109:715) - -### Command Abstraction -The Command Abstraction offers a high-level interface for executing Redis commands, encompassing implementations for various command categories like keys, hashes, lists, sets, and sorted sets. It supports both synchronous and asynchronous execution, providing a consistent API for interacting with Redis regardless of the underlying connection type. This component also manages command encoding and decoding, ensuring seamless communication with the Redis server. -- **Related Classes/Methods**: `redis.commands.core.BasicKeyCommands` (1557:2510), `redis.commands.core.HashCommands` (4921:5598), `redis.commands.core.ListCommands` (2533:2947), `redis.commands.core.SetCommands` (3287:3462), `redis.commands.core.SortedSetCommands` (4077:4870), `redis.commands.core.StreamCommands` (3468:4071), `redis.commands.core.PubSubCommands` (5720:5784), `redis.commands.core.AsyncBasicKeyCommands` (2513:2530) - -### Connection Management -The Connection Management component is responsible for establishing and maintaining connections to the Redis server. It provides connection pooling, socket management, and authentication functionalities. Supporting various connection types, including TCP, SSL, and Unix domain sockets, it also implements retry mechanisms for handling connection errors. This component ensures reliable and efficient communication with the Redis server. -- **Related Classes/Methods**: `redis.connection.ConnectionPool` (1309:1654), `redis.asyncio.connection.ConnectionPool` (1031:1253), `redis.connection.Connection` (730:801), `redis.asyncio.connection.Connection` (723:777) - -### Cluster Support -The Cluster Support component provides functionalities for managing and interacting with Redis clusters, handling node discovery, slot assignment, and command routing. It supports both synchronous and asynchronous cluster operations, offering a consistent API for interacting with Redis clusters. This component is responsible for distributing commands across the cluster and handling failover scenarios, ensuring high availability and scalability. -- **Related Classes/Methods**: `redis.cluster.RedisCluster` (456:1360), `redis.asyncio.cluster.RedisCluster` (99:989), `redis.cluster.NodesManager` (1443:1863), `redis.asyncio.cluster.NodesManager` (1211:1518) - -### PubSub Management -The PubSub Management component provides publish/subscribe functionalities for real-time messaging, enabling clients to subscribe to channels and receive messages published to those channels. It supports both synchronous and asynchronous pub/sub operations, managing subscriptions and distributing messages to subscribers efficiently. This component facilitates real-time communication and event-driven architectures. -- **Related Classes/Methods**: `redis.client.PubSub` (743:1241), `redis.asyncio.client.PubSub` (803:1231), `redis.cluster.ClusterPubSub` (1866:2107) - -### Data Handling -The Data Handling component manages the serialization of commands and parsing of responses between the client and the Redis server. It supports different serialization formats, including RESP2 and RESP3, and includes helper functions for parsing specific data types. This component ensures that data is correctly formatted for transmission to and from the Redis server, maintaining data integrity and compatibility. -- **Related Classes/Methods**: `redis._parsers.encoders.Encoder` (4:44), `redis._parsers.commands.CommandsParser` (56:170), `redis._parsers.commands.AsyncCommandsParser` (173:281), `redis._parsers.helpers` (full file reference) \ No newline at end of file From 25c19d6fc9d0bfa9f9dd67687fe006dcbaeca5d7 Mon Sep 17 00:00:00 2001 From: ivanmilevtues Date: Sat, 5 Jul 2025 17:00:09 +0200 Subject: [PATCH 3/3] Added high-level diagrams --- .codeboarding/Client_API.md | 273 ++++++++++++++++++ .codeboarding/Command & Module Execution.md | 189 ++++++++++++ .../Connection & Protocol Management.md | 173 +++++++++++ .codeboarding/Connection_Management.md | 179 ++++++++++++ .codeboarding/Error Handling.md | 185 ++++++++++++ .../High Availability & Cluster Management.md | 152 ++++++++++ .codeboarding/Protocol_Command_Handling.md | 163 +++++++++++ .codeboarding/Redis Client Core.md | 119 ++++++++ .../Utility_Cross_Cutting_Concerns.md | 201 +++++++++++++ .codeboarding/on_boarding.md | 192 +++++++++--- 10 files changed, 1778 insertions(+), 48 deletions(-) create mode 100644 .codeboarding/Client_API.md create mode 100644 .codeboarding/Connection_Management.md create mode 100644 .codeboarding/Protocol_Command_Handling.md create mode 100644 .codeboarding/Utility_Cross_Cutting_Concerns.md diff --git a/.codeboarding/Client_API.md b/.codeboarding/Client_API.md new file mode 100644 index 0000000000..9c1c62ac07 --- /dev/null +++ b/.codeboarding/Client_API.md @@ -0,0 +1,273 @@ +```mermaid + +graph LR + + Redis_Client_Synchronous_["Redis Client (Synchronous)"] + + Redis_Client_Asynchronous_["Redis Client (Asynchronous)"] + + Connection_Management["Connection Management"] + + Protocol_Handling["Protocol Handling"] + + Command_Implementations["Command Implementations"] + + Error_Handling["Error Handling"] + + Authentication_Security["Authentication & Security"] + + Backoff_Strategies["Backoff Strategies"] + + Event_Management["Event Management"] + + Redis_Client_Synchronous_ -- "uses" --> Connection_Management + + Redis_Client_Synchronous_ -- "uses" --> Command_Implementations + + Redis_Client_Synchronous_ -- "uses" --> Protocol_Handling + + Redis_Client_Synchronous_ -- "uses" --> Error_Handling + + Redis_Client_Asynchronous_ -- "uses" --> Connection_Management + + Redis_Client_Asynchronous_ -- "uses" --> Command_Implementations + + Redis_Client_Asynchronous_ -- "uses" --> Protocol_Handling + + Redis_Client_Asynchronous_ -- "uses" --> Error_Handling + + Connection_Management -- "uses" --> Protocol_Handling + + Connection_Management -- "uses" --> Authentication_Security + + Connection_Management -- "uses" --> Backoff_Strategies + + Connection_Management -- "dispatches events to" --> Event_Management + + Protocol_Handling -- "raises exceptions to" --> Error_Handling + + Command_Implementations -- "uses" --> Protocol_Handling + + Command_Implementations -- "raises errors to" --> Error_Handling + + Authentication_Security -- "raises errors to" --> Error_Handling + + Event_Management -- "triggers" --> Authentication_Security + + click Connection_Management href "https://github.com/redis/redis-py/blob/master/.codeboarding//Connection_Management.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +Analysis of the Client API subsystem in redis-py + + + +### Redis Client (Synchronous) + +This component provides the synchronous, blocking interface for interacting with a Redis server. It exposes methods for all standard Redis commands, allowing users to execute operations directly. It also manages synchronous command pipelines and transactions. + + + + + +**Related Classes/Methods**: + + + +- `redis.client.Redis` (111:667) + +- `redis.client.Pipeline` (1278:1630) + + + + + +### Redis Client (Asynchronous) + +This component offers the asynchronous, non-blocking interface for interacting with Redis, built on `asyncio`. It mirrors the synchronous client's functionality but is designed for high-concurrency applications, enabling efficient I/O operations without blocking the event loop. + + + + + +**Related Classes/Methods**: + + + +- `redis.asyncio.client.Redis` (108:714) + +- `redis.asyncio.client.Pipeline` (1250:1617) + + + + + +### Connection Management [[Expand]](./Connection_Management.md) + +This component is responsible for establishing, maintaining, and pooling connections to Redis servers. It handles the low-level network communication details, including TCP/IP sockets, SSL/TLS, and Unix domain sockets, ensuring efficient resource utilization and connection stability. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.ConnectionPool` (1308:1647) + +- `redis.connection.Connection` (729:800) + +- `redis.asyncio.connection.ConnectionPool` (1030:1252) + +- `redis.asyncio.connection.Connection` (722:776) + + + + + +### Protocol Handling + +This component manages the serialization and deserialization of data according to the Redis Serialization Protocol (RESP). It encodes Python commands into RESP format for transmission to Redis and decodes RESP responses back into Python data types. It supports both RESP2 and RESP3 and can leverage `hiredis` for faster parsing. + + + + + +**Related Classes/Methods**: + + + +- `redis._parsers.base.BaseParser` (53:104) + +- `redis._parsers.hiredis._HiredisParser` (40:183) + +- `redis._parsers.resp2._RESP2Parser` (8:67) + +- `redis._parsers.resp3._RESP3Parser` (14:130) + + + + + +### Command Implementations + +This component provides the concrete implementations for the vast array of Redis commands. Commands are often organized into mixin classes (e.g., `CoreCommands`, `JSONCommands`, `SearchCommands`) that are then inherited by the main `Redis` client classes, promoting modularity and extensibility. + + + + + +**Related Classes/Methods**: + + + +- `redis.commands.core.CoreCommands` (6642:6655) + +- `redis.commands.core.AsyncCoreCommands` (6658:6671) + +- `redis.commands.json.JSON` (1:1) + +- `redis.commands.search.Search` (1:1) + + + + + +### Error Handling + +This component defines a comprehensive hierarchy of custom exception classes specific to Redis operations. It provides a structured and user-friendly way to report various issues, such as connection failures, invalid responses, authentication errors, and cluster-specific problems. + + + + + +**Related Classes/Methods**: + + + +- `redis.exceptions.RedisError` (3:4) + +- `redis.exceptions.ConnectionError` (7:8) + +- `redis.exceptions.ResponseError` (31:32) + + + + + +### Authentication & Security + +This component manages authentication credentials and secure communication settings. It handles mechanisms like password-based authentication, token-based authentication (e.g., JWT), and SSL/TLS configurations to ensure secure connections to Redis servers. + + + + + +**Related Classes/Methods**: + + + +- `redis.auth.token.TokenInterface` (6:29) + +- `redis.credentials.CredentialProvider` (7:20) + + + + + +### Backoff Strategies + +This component provides various algorithms for implementing retry logic with exponential backoff and jitter. It helps in gracefully handling transient network issues or temporary server unavailability by retrying failed operations with increasing delays, preventing overwhelming the server. + + + + + +**Related Classes/Methods**: + + + +- `redis.backoff.AbstractBackoff` (9:23) + +- `redis.backoff.ExponentialBackoff` (53:74) + + + + + +### Event Management + +This component provides a mechanism for dispatching and listening to internal events within the client library. It allows for custom logic to be triggered at specific points in the client's lifecycle, such as after a connection is released or during re-authentication, enabling extensibility and observability. + + + + + +**Related Classes/Methods**: + + + +- `redis.event.EventDispatcher` (56:89) + +- `redis.event.EventListenerInterface` (10:17) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Command & Module Execution.md b/.codeboarding/Command & Module Execution.md index 3d84c2e47f..bf0c21554c 100644 --- a/.codeboarding/Command & Module Execution.md +++ b/.codeboarding/Command & Module Execution.md @@ -1,190 +1,379 @@ ```mermaid + graph LR + RedisCoreCommands["RedisCoreCommands"] + RedisModuleIntegration["RedisModuleIntegration"] + RedisJSONModule["RedisJSONModule"] + RedisSearchModule["RedisSearchModule"] + RedisTimeSeriesModule["RedisTimeSeriesModule"] + RedisBloomModule["RedisBloomModule"] + RedisVectorSetModule["RedisVectorSetModule"] + RedisClusterManagement["RedisClusterManagement"] + RedisScriptingAndPubSub["RedisScriptingAndPubSub"] + RedisCommandHelpers["RedisCommandHelpers"] + RedisModuleIntegration -- "integrates" --> RedisJSONModule + RedisModuleIntegration -- "integrates" --> RedisSearchModule + RedisModuleIntegration -- "integrates" --> RedisTimeSeriesModule + RedisModuleIntegration -- "integrates" --> RedisBloomModule + RedisModuleIntegration -- "integrates" --> RedisVectorSetModule + RedisClusterManagement -- "utilizes" --> RedisCommandHelpers + RedisClusterManagement -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisCoreCommands -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisCoreCommands -- "utilizes" --> RedisCommandHelpers + RedisScriptingAndPubSub -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisScriptingAndPubSub -- "utilizes" --> RedisCommandHelpers + RedisVectorSetModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisVectorSetModule -- "utilizes" --> RedisCommandHelpers + RedisTimeSeriesModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisTimeSeriesModule -- "utilizes" --> RedisCommandHelpers + RedisSearchModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisSearchModule -- "utilizes" --> RedisCommandHelpers + RedisBloomModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisBloomModule -- "utilizes" --> RedisCommandHelpers + RedisJSONModule -- "executes commands via" --> redis_typing_CommandsProtocol_execute_command + RedisJSONModule -- "utilizes" --> RedisCommandHelpers + ``` + [![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + ## Component Details + + This component overview describes the structure, flow, and purpose of the Command & Module Execution subsystem within the Redis client library. It focuses on how standard Redis commands are implemented and executed, and how interactions with various Redis Modules (e.g., JSON, Search, TimeSeries) are facilitated. The system provides a comprehensive interface for both core Redis functionalities and extended module capabilities, ensuring efficient data operations and server management. + + ### RedisCoreCommands + This component provides the fundamental Redis commands for interacting with various data structures (strings, lists, sets, sorted sets, hashes, streams, geo-spatial) and handles administrative and management commands for the Redis server (configuration, client management, server-wide operations, ACL, modules, cluster commands). It serves as the primary interface for basic Redis operations. + + + **Related Classes/Methods**: + + - `redis.commands.core.ACLCommands` (full file reference) + - `redis.commands.core.ManagementCommands` (full file reference) + - `redis.commands.core.AsyncManagementCommands` (full file reference) + - `redis.commands.core.BitFieldOperation` (full file reference) + - `redis.commands.core.BasicKeyCommands` (full file reference) + - `redis.commands.core.AsyncBasicKeyCommands` (full file reference) + - `redis.commands.core.ListCommands` (full file reference) + - `redis.commands.core.ScanCommands` (full file reference) + - `redis.commands.core.AsyncScanCommands` (full file reference) + - `redis.commands.core.SetCommands` (full file reference) + - `redis.commands.core.StreamCommands` (full file reference) + - `redis.commands.core.SortedSetCommands` (full file reference) + - `redis.commands.core.HyperlogCommands` (full file reference) + - `redis.commands.core.HashCommands` (full file reference) + - `redis.commands.core.GeoCommands` (full file reference) + - `redis.commands.core.ModuleCommands` (full file reference) + - `redis.commands.core.AsyncModuleCommands` (full file reference) + - `redis.commands.core.ClusterCommands` (full file reference) + + + ### RedisModuleIntegration + This component acts as an integration layer for various Redis modules, providing a unified interface to access their functionalities. It serves as a central point for dispatching commands to specific module clients. + + + **Related Classes/Methods**: + + - `redis.commands.redismodules.RedisModuleCommands` (14:91) + - `redis.commands.redismodules.AsyncRedisModuleCommands` (94:101) + + + ### RedisJSONModule + This component offers a client for the RedisJSON module, allowing for efficient storage and manipulation of JSON documents within Redis. It provides methods for JSON-specific operations like setting, getting, and manipulating JSON paths. + + + **Related Classes/Methods**: + + - `redis.commands.json.commands.JSONCommands` (13:431) + - `redis.commands.json.JSON` (full file reference) + + + ### RedisSearchModule + This component provides a comprehensive client for the Redis Search module, supporting index creation, document management, complex query building, and aggregation. It enables full-text search capabilities within Redis. + + + **Related Classes/Methods**: + + - `redis.commands.search.index_definition.IndexDefinition` (11:79) + - `redis.commands.search.aggregation.AggregateRequest` (89:372) + - `redis.commands.search.field.TextField` (79:109) + - `redis.commands.search.field.NumericField` (112:118) + - `redis.commands.search.field.GeoShapeField` (121:133) + - `redis.commands.search.field.GeoField` (136:142) + - `redis.commands.search.field.TagField` (145:168) + - `redis.commands.search.field.VectorField` (171:210) + - `redis.commands.search.commands.SearchCommands` (full file reference) + - `redis.commands.search.commands.AsyncSearchCommands` (full file reference) + - `redis.commands.search.query.Query` (6:339) + - `redis.commands.search.query.NumericFilter` (347:364) + - `redis.commands.search.query.GeoFilter` (367:376) + - `redis.commands.search.querystring` (316:317) + - `redis.commands.search.reducers` (full file reference) + - `redis.commands.search.suggestion.Suggestion` (6:20) + - `redis.commands.search.suggestion.SuggestionParser` (23:55) + - `redis.commands.search.result.Result` (7:87) + - `redis.commands.search.Search` (full file reference) + - `redis.commands.search.AsyncSearch` (full file reference) + + + ### RedisTimeSeriesModule + This component offers a client for the Redis TimeSeries module, allowing for the creation, manipulation, and querying of time-series data. It provides functionalities for adding samples, querying ranges, and managing time-series data. + + + **Related Classes/Methods**: + + - `redis.commands.timeseries.info.TSInfo` (5:91) + - `redis.commands.timeseries.commands.TimeSeriesCommands` (25:1000) + - `redis.commands.timeseries.TimeSeries` (full file reference) + + + ### RedisBloomModule + This component provides client-side access to the RedisBloom module, enabling the use of probabilistic data structures like Bloom filters, Cuckoo filters, Count-Min sketches, and TopK. It offers methods for interacting with these specialized data structures. + + + **Related Classes/Methods**: + + - `redis.commands.bf.info.BFInfo` (4:26) + - `redis.commands.bf.info.CFInfo` (29:57) + - `redis.commands.bf.info.TDigestInfo` (92:120) + - `redis.commands.bf.commands.TOPKCommands` (292:356) + - `redis.commands.bf.CMSBloom` (full file reference) + - `redis.commands.bf.TOPKBloom` (full file reference) + - `redis.commands.bf.CFBloom` (full file reference) + - `redis.commands.bf.TDigestBloom` (full file reference) + - `redis.commands.bf.BFBloom` (full file reference) + + + ### RedisVectorSetModule + This component provides the client-side interface for interacting with the Redis VectorSet module, enabling vector similarity search and related operations. It allows for storing and querying vector data within Redis. + + + **Related Classes/Methods**: + + - `redis.commands.vectorset.commands.VectorSetCommands` (40:367) + - `redis.commands.vectorset.VectorSet` (full file reference) + + + ### RedisClusterManagement + This component is responsible for managing Redis Cluster specific operations, including multi-key commands, node management, and data partitioning across slots. It provides an interface for interacting with a Redis Cluster setup. + + + **Related Classes/Methods**: + + - `redis.commands.cluster.ClusterMultiKeyCommands` (99:260) + - `redis.commands.cluster.AsyncClusterMultiKeyCommands` (263:339) + - `redis.commands.cluster.ClusterManagementCommands` (342:692) + - `redis.commands.cluster.AsyncClusterManagementCommands` (695:719) + - `redis.commands.cluster.ClusterDataAccessCommands` (722:810) + + + ### RedisScriptingAndPubSub + This component encapsulates functionalities related to Redis scripting (Lua scripts) and Publish/Subscribe messaging. It provides methods for executing scripts, managing script caches, and handling pub/sub operations. + + + **Related Classes/Methods**: + + - `redis.commands.core.Script` (full file reference) + - `redis.commands.core.PubSubCommands` (full file reference) + - `redis.commands.core.ScriptCommands` (full file reference) + - `redis.commands.core.AsyncScriptCommands` (full file reference) + - `redis.commands.core.FunctionCommands` (full file reference) + + + ### RedisCommandHelpers + This component provides various utility functions and helper methods that are commonly used across different Redis command implementations for tasks like argument parsing, data conversion, and protocol version handling. It supports the other command components by offering shared functionalities. + + + **Related Classes/Methods**: + + - `redis.commands.helpers` (full file reference) + + + + + ### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Connection & Protocol Management.md b/.codeboarding/Connection & Protocol Management.md index e40d2710f5..26ee5c68c3 100644 --- a/.codeboarding/Connection & Protocol Management.md +++ b/.codeboarding/Connection & Protocol Management.md @@ -1,174 +1,347 @@ ```mermaid + graph LR + Core_Connection_Management["Core Connection Management"] + Connection_Pooling["Connection Pooling"] + Protocol_Parsers["Protocol Parsers"] + Authentication_Security["Authentication & Security"] + Error_Handling["Error Handling"] + Client_Interfaces["Client Interfaces"] + Utility_Eventing["Utility & Eventing"] + Retry_Backoff["Retry & Backoff"] + Caching["Caching"] + Client_Interfaces -- "uses" --> Connection_Pooling + Connection_Pooling -- "manages" --> Core_Connection_Management + Core_Connection_Management -- "uses" --> Protocol_Parsers + Core_Connection_Management -- "raises" --> Error_Handling + Core_Connection_Management -- "integrates" --> Authentication_Security + Core_Connection_Management -- "applies" --> Retry_Backoff + Protocol_Parsers -- "raises" --> Error_Handling + Authentication_Security -- "raises" --> Error_Handling + Connection_Pooling -- "notifies" --> Utility_Eventing + Connection_Pooling -- "leverages" --> Caching + Client_Interfaces -- "utilizes" --> Caching + Utility_Eventing -- "supports" --> Core_Connection_Management + Utility_Eventing -- "supports" --> Protocol_Parsers + ``` + [![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + ## Component Details + + The Connection & Protocol Management subsystem in the Redis client library is a comprehensive system designed to facilitate robust and efficient communication with Redis servers. It manages the entire lifecycle of connections, from establishment and pooling to secure authentication and error handling. The subsystem handles the intricacies of Redis Serialization Protocols (RESP2 and RESP3) for command encoding and response parsing, ensuring data integrity and performance. It also incorporates caching mechanisms, retry strategies for transient failures, and provides high-level client interfaces that abstract the underlying complexities, enabling seamless interaction for users. + + ### Core Connection Management + This component encapsulates the fundamental logic for establishing, maintaining, and managing connections to Redis servers, including both synchronous and asynchronous implementations. It handles various connection types such as standard TCP, SSL, and Unix domain sockets, and manages the lifecycle of individual connections. + + + **Related Classes/Methods**: + + - `redis.connection.AbstractConnection` (full file reference) + - `redis.connection.Connection` (full file reference) + - `redis.connection.SSLConnection` (full file reference) + - `redis.connection.UnixDomainSocketConnection` (full file reference) + - `redis.connection.CacheProxyConnection` (full file reference) + - `redis.asyncio.connection.AbstractConnection` (106:720) + - `redis.asyncio.connection.Connection` (723:777) + - `redis.asyncio.connection.SSLConnection` (780:844) + - `redis.asyncio.connection.UnixDomainSocketConnection` (916:937) + + + ### Connection Pooling + This component is responsible for efficiently managing a pool of Redis connections. It optimizes resource utilization by reusing existing connections, thereby reducing the overhead of establishing new connections for each operation. It provides mechanisms for acquiring and releasing connections, and supports both blocking and non-blocking behaviors. + + + **Related Classes/Methods**: + + - `redis.connection.ConnectionPool` (full file reference) + - `redis.connection.BlockingConnectionPool` (full file reference) + - `redis.asyncio.connection.ConnectionPool` (full file reference) + - `redis.asyncio.connection.BlockingConnectionPool` (full file reference) + + + ### Protocol Parsers + This component handles the serialization of commands sent to Redis and the deserialization of responses received from the server. It supports different Redis Serialization Protocols (RESP2 and RESP3) and can leverage optimized C implementations like hiredis for improved performance. It also includes logic for parsing command structures and handling socket buffer operations. + + + **Related Classes/Methods**: + + - `redis._parsers.encoders.Encoder` (4:44) + - `redis._parsers.socket.SocketBuffer` (29:162) + - `redis._parsers.commands.AbstractCommandsParser` (10:53) + - `redis._parsers.commands.CommandsParser` (56:170) + - `redis._parsers.commands.AsyncCommandsParser` (173:281) + - `redis._parsers.hiredis._HiredisParser` (41:184) + - `redis._parsers.hiredis._AsyncHiredisParser` (187:295) + - `redis._parsers.resp3._RESP3Parser` (15:131) + - `redis._parsers.resp3._AsyncRESP3Parser` (134:257) + - `redis._parsers.base.BaseParser` (54:105) + - `redis._parsers.base._RESPBase` (108:140) + - `redis._parsers.base._AsyncRESPBase` (213:289) + - `redis._parsers.resp2._RESP2Parser` (9:68) + - `redis._parsers.resp2._AsyncRESP2Parser` (71:132) + + + ### Authentication & Security + This component provides functionalities for authenticating with Redis servers and ensuring secure communication. It includes mechanisms for managing authentication tokens, such as JWT, and implements OCSP (Online Certificate Status Protocol) verification to validate SSL certificates, enhancing the security posture of connections. + + + **Related Classes/Methods**: + + - `redis.auth.token_manager.TokenManager` (121:340) + - `redis.auth.token.SimpleToken` (44:75) + - `redis.auth.token.JWToken` (78:130) + - `redis.ocsp._verify_response` (22:47) + - `redis.ocsp._check_certificate` (50:106) + - `redis.ocsp._get_certificates` (109:123) + - `redis.ocsp.ocsp_staple_verifier` (142:167) + - `redis.ocsp.OCSPVerifier` (170:308) + - `redis.asyncio.connection.RedisSSLContext` (847:913) + + + ### Error Handling + This component defines a comprehensive set of custom exception classes that represent various error conditions encountered during Redis operations, such as connection failures, timeouts, authentication issues, and invalid responses. It centralizes error management, allowing for more specific and robust error handling throughout the library. + + + **Related Classes/Methods**: + + - `redis.exceptions.DataError` (36:37) + - `redis.exceptions.ConnectionError` (8:9) + - `redis.exceptions.TimeoutError` (12:13) + - `redis.exceptions.AuthenticationError` (16:17) + - `redis.exceptions.RedisError` (4:5) + - `redis.exceptions.InvalidResponse` (28:29) + - `redis.exceptions.ResponseError` (32:33) + - `redis.auth.err.TokenRenewalErr` (25:31) + - `redis.auth.err.InvalidTokenSchemaErr` (13:22) + - `redis.exceptions.AuthorizationError` (20:21) + + + ### Client Interfaces + This component provides the high-level API for users to interact with Redis. It includes the standard Redis client, as well as specialized clients for Redis Cluster and Redis Sentinel, abstracting the underlying connection and protocol details to offer a user-friendly interface for executing Redis commands. + + + **Related Classes/Methods**: + + - `redis.cluster.RedisCluster` (456:1000) + - `redis.cluster.NodesManager` (full file reference) + - `redis.sentinel.SentinelConnectionPool` (137:195) + - `redis.client.Redis` (112:670) + + + ### Utility & Eventing + This component comprises a collection of general-purpose utility functions that support various operations across the library, such as string manipulation, version comparison, and argument deprecation handling. It also includes an event dispatching mechanism that allows different parts of the system to communicate and react to specific events, like connection releases. + + + **Related Classes/Methods**: + + - `redis.utils.get_lib_version` (211:216) + - `redis.utils.format_error_message` (219:228) + - `redis.utils.str_if_bytes` (60:63) + - `redis.utils.deprecated_args` (153:195) + - `redis.utils.ensure_string` (261:267) + - `redis.utils.compare_versions` (231:258) + - `redis.event.EventDispatcher` (57:90) + - `redis.event.AfterConnectionReleasedEvent` (93:103) + - `redis.event.AsyncAfterConnectionReleasedEvent` (106:107) + + + ### Retry & Backoff + This component implements robust strategies for handling transient failures by retrying operations with configurable delays. It includes mechanisms for exponential backoff and allows for the definition of supported error types, ensuring that the client can gracefully recover from temporary network issues or server unavailability. + + + **Related Classes/Methods**: + + - `redis.retry.Retry` (13:95) + - `redis.backoff.NoBackoff` (47:51) + - `redis.asyncio.retry.Retry` (13:79) + + + ### Caching + This component provides an internal caching mechanism used to store and retrieve frequently accessed data, such as command information or connection details, to improve performance. It includes different caching policies, like LRU, and a factory for creating cache instances. + + + **Related Classes/Methods**: + + - `redis.cache.DefaultCache` (142:224) + - `redis.cache.LRUPolicy` (227:271) + - `redis.cache.CacheFactory` (392:401) + - `redis.cache.CacheConfig` (278:383) + - `redis.cache.CacheKey` (19:21) + - `redis.cache.CacheEntry` (24:43) + + + + + ### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Connection_Management.md b/.codeboarding/Connection_Management.md new file mode 100644 index 0000000000..f6cc50977c --- /dev/null +++ b/.codeboarding/Connection_Management.md @@ -0,0 +1,179 @@ +```mermaid + +graph LR + + Connection_Abstraction["Connection Abstraction"] + + Connection_Pooling["Connection Pooling"] + + Sentinel_Integration["Sentinel Integration"] + + Cluster_Integration["Cluster Integration"] + + Exception_Handling["Exception Handling"] + + Connection_Abstraction -- "uses" --> Exception_Handling + + Connection_Pooling -- "uses" --> Connection_Abstraction + + Sentinel_Integration -- "relies on" --> Connection_Pooling + + Sentinel_Integration -- "uses" --> Exception_Handling + + Cluster_Integration -- "relies on" --> Connection_Pooling + + Cluster_Integration -- "uses" --> Exception_Handling + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The Connection Management subsystem in redis-py is crucial for establishing and maintaining connections to various Redis deployments. It abstracts the complexities of network communication, connection pooling, and handling different Redis topologies (standalone, Sentinel, Cluster). + + + +### Connection Abstraction + +This component provides the foundational classes for managing individual connections to a Redis server. It handles the low-level details of establishing TCP or Unix socket connections, sending commands, and receiving responses. It also includes support for SSL/TLS connections. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.Connection` (729:800) + +- `redis.connection.SSLConnection` (1017:1185) + +- `redis.connection.UnixDomainSocketConnection` (1188:1220) + +- `redis.asyncio.connection.Connection` (722:776) + +- `redis.asyncio.connection.SSLConnection` (779:843) + +- `redis.asyncio.connection.UnixDomainSocketConnection` (915:936) + + + + + +### Connection Pooling + +This component manages a pool of connections to Redis, improving performance by reusing existing connections instead of establishing new ones for each operation. It handles connection lifecycle, including creation, release, and optional blocking behavior when the pool is exhausted. + + + + + +**Related Classes/Methods**: + + + +- `redis.connection.ConnectionPool` (1308:1647) + +- `redis.connection.BlockingConnectionPool` (1650:1817) + +- `redis.asyncio.connection.ConnectionPool` (1030:1252) + +- `redis.asyncio.connection.BlockingConnectionPool` (1255:1332) + + + + + +### Sentinel Integration + +This component provides support for connecting to and managing Redis Sentinel deployments. It enables the client to discover the current master and replica nodes, handle failovers, and automatically re-route commands to the correct master. + + + + + +**Related Classes/Methods**: + + + +- `redis.sentinel.Sentinel` (197:420) + +- `redis.sentinel.SentinelConnectionPool` (136:194) + +- `redis.sentinel.SentinelManagedConnection` (19:81) + +- `redis.asyncio.sentinel.Sentinel` (166:400) + +- `redis.asyncio.sentinel.SentinelConnectionPool` (95:163) + +- `redis.asyncio.sentinel.SentinelManagedConnection` (25:88) + + + + + +### Cluster Integration + +This component handles connections and command routing within a Redis Cluster environment. It manages slot mapping, handles redirection errors (MOVED/ASK), and discovers cluster nodes, ensuring commands are sent to the appropriate shard. + + + + + +**Related Classes/Methods**: + + + +- `redis.cluster.RedisCluster` (455:1358) + +- `redis.cluster.NodesManager` (1441:1861) + +- `redis.asyncio.cluster.RedisCluster` (98:988) + +- `redis.asyncio.cluster.NodesManager` (1210:1517) + + + + + +### Exception Handling + +This component defines a hierarchy of custom exceptions specific to Redis operations and connection issues. It provides a structured way to handle errors such as connection failures, timeouts, authentication errors, and Redis-specific response errors. + + + + + +**Related Classes/Methods**: + + + +- `redis.exceptions.RedisError` (3:4) + +- `redis.exceptions.ConnectionError` (7:8) + +- `redis.exceptions.TimeoutError` (11:12) + +- `redis.exceptions.AuthenticationError` (15:16) + +- `redis.exceptions.ResponseError` (31:32) + +- `redis.exceptions.ClusterError` (119:125) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Error Handling.md b/.codeboarding/Error Handling.md index 8befc06fa6..26e3654a7b 100644 --- a/.codeboarding/Error Handling.md +++ b/.codeboarding/Error Handling.md @@ -1,186 +1,371 @@ ```mermaid + graph LR + Error_Handling["Error Handling"] + Redis_Client["Redis Client"] + Connection_Management["Connection Management"] + Redis_Cluster_Management["Redis Cluster Management"] + Distributed_Locking["Distributed Locking"] + Redis_Sentinel_Integration["Redis Sentinel Integration"] + Protocol_Parsing["Protocol Parsing"] + Authentication_and_Security["Authentication and Security"] + Redis_Client -- "utilizes" --> Connection_Management + Redis_Client -- "interprets responses via" --> Protocol_Parsing + Redis_Client -- "raises" --> Error_Handling + Connection_Management -- "relies on" --> Protocol_Parsing + Connection_Management -- "raises" --> Error_Handling + Redis_Cluster_Management -- "manages connections through" --> Connection_Management + Redis_Cluster_Management -- "raises" --> Error_Handling + Distributed_Locking -- "leverages" --> Redis_Client + Distributed_Locking -- "raises" --> Error_Handling + Redis_Sentinel_Integration -- "manages connections through" --> Connection_Management + Redis_Sentinel_Integration -- "raises" --> Error_Handling + Protocol_Parsing -- "raises" --> Error_Handling + Authentication_and_Security -- "configures" --> Connection_Management + ``` + [![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + ## Component Details + + This architecture overview describes the core components of the redis-py library, focusing on how it manages connections, handles Redis commands, interacts with Redis clusters and Sentinels, and provides robust error handling and security features. The main flow involves clients initiating connections through connection management, sending commands which are then parsed and executed, and receiving responses that are processed by the protocol parsing component. Error handling is integrated throughout, providing specific exceptions for various operational failures. Asynchronous capabilities are seamlessly integrated into the core components, offering non-blocking operations for high-performance applications. + + ### Error Handling + Defines and manages custom exception classes for various Redis-related errors, providing a structured and specific way to handle different error scenarios that can occur during client-server interactions. + + + **Related Classes/Methods**: + + - `redis.exceptions.RedisError` (4:5) + - `redis.exceptions.ConnectionError` (8:9) + - `redis.exceptions.AuthenticationError` (16:17) + - `redis.exceptions.AuthorizationError` (20:21) + - `redis.exceptions.BusyLoadingError` (24:25) + - `redis.exceptions.MaxConnectionsError` (223:223) + - `redis.exceptions.TimeoutError` (12:13) + - `redis.exceptions.DataError` (36:37) + - `redis.exceptions.PubSubError` (40:41) + - `redis.exceptions.InvalidResponse` (28:29) + - `redis.exceptions.ResponseError` (32:33) + - `redis.exceptions.AuthenticationWrongNumberOfArgsError` (103:109) + - `redis.exceptions.NoPermissionError` (72:73) + - `redis.exceptions.ExecAbortError` (64:65) + - `redis.exceptions.ReadOnlyError` (68:69) + - `redis.exceptions.NoScriptError` (48:49) + - `redis.exceptions.OutOfMemoryError` (52:61) + - `redis.exceptions.ModuleError` (76:77) + - `redis.exceptions.WatchError` (44:45) + - `redis.exceptions.LockError` (80:88) + - `redis.exceptions.LockNotOwnedError` (91:94) + - `redis.exceptions.TryAgainError` (171:179) + - `redis.exceptions.ClusterError` (120:126) + - `redis.exceptions.ClusterDownError` (129:142) + - `redis.exceptions.MasterDownError` (202:208) + - `redis.exceptions.AskError` (145:168) + - `redis.exceptions.MovedError` (192:199) + - `redis.exceptions.ClusterCrossSlotError` (182:189) + - `redis.exceptions.RedisClusterException` (112:117) + - `redis.exceptions.SlotNotCoveredError` (211:220) + - `redis.exceptions.CrossSlotTransactionError` (226:232) + - `redis.exceptions.InvalidPipelineStack` (235:241) + + + ### Redis Client + Provides the fundamental synchronous and asynchronous interfaces for interacting with a single Redis instance. It includes functionalities for executing commands, managing Pub/Sub subscriptions, and handling transactional pipelines. + + + **Related Classes/Methods**: + + - `redis.client.Redis` (112:670) + - `redis.client.Monitor` (676:740) + - `redis.client.PubSub` (743:1000) + - `redis.client.Pipeline` (full file reference) + - `redis.asyncio.client.Redis` (full file reference) + - `redis.asyncio.client.Monitor` (full file reference) + - `redis.asyncio.client.PubSub` (full file reference) + - `redis.asyncio.client.Pipeline` (full file reference) + + + ### Connection Management + Responsible for establishing, maintaining, and pooling synchronous and asynchronous connections to Redis servers. It handles connection health checks, error handling during connection attempts, and SSL/TLS wrapping. + + + **Related Classes/Methods**: + + - `redis.connection.HiredisRespSerializer` (full file reference) + - `redis.connection.AbstractConnection` (full file reference) + - `redis.connection.CacheProxyConnection` (full file reference) + - `redis.connection.SSLConnection` (full file reference) + - `redis.connection.ConnectionPool` (full file reference) + - `redis.connection.BlockingConnectionPool` (full file reference) + - `redis.asyncio.connection.AbstractConnection` (106:720) + - `redis.asyncio.connection.SSLConnection` (780:844) + - `redis.asyncio.connection.RedisSSLContext` (847:913) + - `redis.asyncio.connection.ConnectionPool` (full file reference) + - `redis.asyncio.connection.BlockingConnectionPool` (full file reference) + + + ### Redis Cluster Management + Manages interactions with a Redis Cluster for both synchronous and asynchronous operations. It handles node discovery, slot mapping, command routing, and error handling specific to a clustered environment, including transaction and pipeline strategies within the cluster. + + + **Related Classes/Methods**: + + - `redis.cluster.RedisCluster` (456:1000) + - `redis.cluster.NodesManager` (full file reference) + - `redis.cluster.ClusterPubSub` (full file reference) + - `redis.cluster.AbstractStrategy` (full file reference) + - `redis.cluster.PipelineStrategy` (full file reference) + - `redis.cluster.TransactionStrategy` (full file reference) + - `redis.commands.cluster.ClusterManagementCommands` (342:692) + - `redis.asyncio.cluster.RedisCluster` (full file reference) + - `redis.asyncio.cluster.ClusterNode` (full file reference) + - `redis.asyncio.cluster.NodesManager` (full file reference) + - `redis.asyncio.cluster.PipelineStrategy` (full file reference) + - `redis.asyncio.cluster.TransactionStrategy` (full file reference) + + + ### Distributed Locking + Provides mechanisms for implementing distributed locks using Redis, ensuring atomicity and proper release of locks for both synchronous and asynchronous contexts. + + + **Related Classes/Methods**: + + - `redis.lock.Lock` (14:343) + - `redis.asyncio.lock.Lock` (17:334) + + + ### Redis Sentinel Integration + Facilitates interaction with Redis Sentinel for high availability, allowing clients to discover master and slave nodes and handle failovers for both synchronous and asynchronous operations. + + + **Related Classes/Methods**: + + - `redis.sentinel.SentinelManagedConnection` (20:82) + - `redis.sentinel.SentinelConnectionPoolProxy` (89:134) + - `redis.sentinel.Sentinel` (198:410) + - `redis.asyncio.sentinel.SentinelManagedConnection` (26:89) + - `redis.asyncio.sentinel.SentinelConnectionPool` (96:164) + - `redis.asyncio.sentinel.Sentinel` (167:389) + + + ### Protocol Parsing + Responsible for parsing responses from the Redis server according to the RESP protocol, including handling different RESP versions and error responses for both synchronous and asynchronous contexts. + + + **Related Classes/Methods**: + + - `redis._parsers.encoders.Encoder` (4:44) + - `redis._parsers.socket.SocketBuffer` (29:162) + - `redis._parsers.commands.CommandsParser` (56:170) + - `redis._parsers.commands.AsyncCommandsParser` (173:281) + - `redis._parsers.hiredis._HiredisParser` (41:184) + - `redis._parsers.hiredis._AsyncHiredisParser` (187:295) + - `redis._parsers.resp3._RESP3Parser` (15:131) + - `redis._parsers.resp3._AsyncRESP3Parser` (134:257) + - `redis._parsers.base.BaseParser` (54:105) + - `redis._parsers.base._AsyncRESPBase` (213:289) + - `redis._parsers.resp2._RESP2Parser` (9:68) + - `redis._parsers.resp2._AsyncRESP2Parser` (71:132) + + + ### Authentication and Security + Handles authentication mechanisms, including token management and OCSP verification for secure connections. + + + **Related Classes/Methods**: + + - `redis.ocsp._verify_response` (22:47) + - `redis.ocsp._check_certificate` (50:106) + - `redis.ocsp.ocsp_staple_verifier` (142:167) + - `redis.ocsp.OCSPVerifier` (170:308) + - `redis.auth.token_manager.TokenManager` (121:340) + - `redis.auth.token.JWToken` (78:130) + + + + + ### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/High Availability & Cluster Management.md b/.codeboarding/High Availability & Cluster Management.md index b972d6c24f..eb44e6abf8 100644 --- a/.codeboarding/High Availability & Cluster Management.md +++ b/.codeboarding/High Availability & Cluster Management.md @@ -1,153 +1,305 @@ ```mermaid + graph LR + Redis_Cluster_Client["Redis Cluster Client"] + Cluster_Node_Manager["Cluster Node Manager"] + Cluster_Node_Representation["Cluster Node Representation"] + Load_Balancer["Load Balancer"] + Cluster_Pub_Sub["Cluster Pub/Sub"] + Cluster_Pipeline["Cluster Pipeline"] + Pipeline_Strategy["Pipeline Strategy"] + Transaction_Strategy["Transaction Strategy"] + Redis_Sentinel_Client["Redis Sentinel Client"] + Sentinel_Connection_Pool["Sentinel Connection Pool"] + Redis_Cluster_Client -- "manages" --> Cluster_Node_Manager + Redis_Cluster_Client -- "creates" --> Cluster_Pipeline + Redis_Cluster_Client -- "creates" --> Cluster_Pub_Sub + Cluster_Node_Manager -- "provides nodes to" --> Redis_Cluster_Client + Cluster_Node_Manager -- "manages" --> Cluster_Node_Representation + Cluster_Node_Manager -- "uses" --> Load_Balancer + Cluster_Pipeline -- "delegates execution to" --> Pipeline_Strategy + Cluster_Pipeline -- "delegates execution to" --> Transaction_Strategy + Redis_Sentinel_Client -- "uses" --> Sentinel_Connection_Pool + Sentinel_Connection_Pool -- "provides connections to" --> Redis_Sentinel_Client + ``` + [![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + ## Component Details + + This graph illustrates the architecture of the High Availability & Cluster Management subsystem, focusing on how Redis Cluster and Redis Sentinel functionalities are provided. It details the components responsible for node discovery, command routing, pipeline execution, and high-availability management in both cluster and sentinel environments. + + ### Redis Cluster Client + The Redis Cluster Client (`RedisCluster`) serves as the primary interface for applications to interact with a Redis Cluster. It handles command routing to appropriate nodes, manages retries on MOVED/ASK errors, and provides methods for obtaining pipeline and pub/sub instances. It supports both synchronous and asynchronous operations. + + + **Related Classes/Methods**: + + - `redis.cluster.RedisCluster` (456:1000) + - `redis.cluster.RedisCluster:__init__` (513:712) + - `redis.cluster.RedisCluster:execute_command` (full file reference) + - `redis.cluster.RedisCluster:pipeline` (841:866) + - `redis.cluster.RedisCluster:pubsub` (834:839) + - `redis.asyncio.cluster.RedisCluster:__init__` (full file reference) + - `redis.asyncio.cluster.RedisCluster:execute_command` (full file reference) + - `redis.asyncio.cluster.RedisCluster:pipeline` (full file reference) + + + ### Cluster Node Manager + The Cluster Node Manager (`NodesManager`) is responsible for discovering and maintaining the topology of the Redis Cluster. It manages the cache of active nodes and their assigned slots, handles updates for MOVED exceptions, and initializes Redis connections to cluster nodes. It also integrates with a load balancer for read operations. + + + **Related Classes/Methods**: + + - `redis.cluster.NodesManager:__init__` (full file reference) + - `redis.cluster.NodesManager:initialize` (full file reference) + - `redis.cluster.NodesManager:get_node_from_slot` (full file reference) + - `redis.asyncio.cluster.NodesManager:__init__` (full file reference) + - `redis.asyncio.cluster.NodesManager:initialize` (full file reference) + - `redis.asyncio.cluster.NodesManager:get_node_from_slot` (full file reference) + + + ### Cluster Node Representation + The Cluster Node Representation (`ClusterNode`) encapsulates the details of a single Redis instance within the cluster, including its host, port, name, and server type (primary or replica). It also holds a reference to the Redis connection object for that specific node. + + + **Related Classes/Methods**: + + - `redis.cluster.ClusterNode:__init__` (full file reference) + - `redis.asyncio.cluster.ClusterNode:__init__` (full file reference) + + + ### Load Balancer + The Load Balancer (`LoadBalancer`) provides strategies for distributing read requests across multiple replica nodes associated with a primary. It supports round-robin and random replica selection to optimize read performance and distribute load. + + + **Related Classes/Methods**: + + - `redis.cluster.LoadBalancer:get_server_index` (full file reference) + + + ### Cluster Pub/Sub + The Cluster Pub/Sub (`ClusterPubSub`) component extends the standard Redis Pub/Sub functionality to work within a Redis Cluster environment. It manages the pub/sub connection to a specific cluster node, determined by channel keyslots or a random node. + + + **Related Classes/Methods**: + + - `redis.cluster.ClusterPubSub:__init__` (full file reference) + - `redis.cluster.ClusterPubSub:execute_command` (full file reference) + - `redis.asyncio.cluster.ClusterPubSub:__init__` (full file reference) + + + ### Cluster Pipeline + The Cluster Pipeline (`ClusterPipeline`) enables batching of multiple Redis commands for efficient execution within a Redis Cluster. It collects commands and then delegates their execution to specific strategies (PipelineStrategy or TransactionStrategy) based on whether a transaction is involved. + + + **Related Classes/Methods**: + + - `redis.cluster.ClusterPipeline:__init__` (full file reference) + - `redis.cluster.ClusterPipeline:execute` (full file reference) + - `redis.asyncio.cluster.ClusterPipeline:__init__` (full file reference) + - `redis.asyncio.cluster.ClusterPipeline:execute` (full file reference) + + + ### Pipeline Strategy + The Pipeline Strategy (`PipelineStrategy`) defines how a batch of commands collected by the Cluster Pipeline is executed across the Redis Cluster. It handles routing commands to the correct nodes and processing their responses, without transactional guarantees. + + + **Related Classes/Methods**: + + - `redis.cluster.PipelineStrategy:__init__` (full file reference) + - `redis.cluster.PipelineStrategy:execute` (full file reference) + - `redis.asyncio.cluster.PipelineStrategy:__init__` (full file reference) + - `redis.asyncio.cluster.PipelineStrategy:execute` (full file reference) + + + ### Transaction Strategy + The Transaction Strategy (`TransactionStrategy`) implements the logic for executing Redis transactions (MULTI/EXEC) within a Redis Cluster. It ensures that all commands within a transaction are sent to the same node and handles WATCH errors and retries. + + + **Related Classes/Methods**: + + - `redis.cluster.TransactionStrategy:__init__` (full file reference) + - `redis.cluster.TransactionStrategy:execute` (full file reference) + - `redis.asyncio.cluster.TransactionStrategy:__init__` (full file reference) + - `redis.asyncio.cluster.TransactionStrategy:execute` (full file reference) + + + ### Redis Sentinel Client + The Redis Sentinel Client (`Sentinel`) provides an interface for applications to connect to Redis instances managed by Sentinel for high availability. It discovers the current master and available replicas for a given service name. + + + **Related Classes/Methods**: + + - `redis.sentinel.Sentinel` (198:410) + - `redis.sentinel.Sentinel:__init__` (227:249) + - `redis.sentinel.Sentinel:master_for` (343:379) + - `redis.asyncio.sentinel.Sentinel:__init__` (196:218) + - `redis.asyncio.sentinel.Sentinel:master_for` (320:357) + + + ### Sentinel Connection Pool + The Sentinel Connection Pool (`SentinelConnectionPool` and `SentinelManagedConnection`) manages the underlying network connections to Redis master and replica instances discovered via Sentinel. It ensures efficient connection reuse and handles connection lifecycle. + + + **Related Classes/Methods**: + + - `redis.sentinel.SentinelConnectionPool:__init__` (145:166) + - `redis.sentinel.SentinelManagedConnection:__init__` (21:23) + - `redis.asyncio.sentinel.SentinelConnectionPool:__init__` (104:120) + - `redis.asyncio.sentinel.SentinelManagedConnection:__init__` (27:29) + + + + + ### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Protocol_Command_Handling.md b/.codeboarding/Protocol_Command_Handling.md new file mode 100644 index 0000000000..fc2edd1376 --- /dev/null +++ b/.codeboarding/Protocol_Command_Handling.md @@ -0,0 +1,163 @@ +```mermaid + +graph LR + + Protocol_Serialization_Deserialization["Protocol Serialization & Deserialization"] + + Redis_Command_Abstraction_Implementation["Redis Command Abstraction & Implementation"] + + Core_Client_API["Core Client API"] + + Connection_Management["Connection Management"] + + Utility_Error_Handling["Utility & Error Handling"] + + Redis_Command_Abstraction_Implementation -- "uses" --> Protocol_Serialization_Deserialization + + Core_Client_API -- "uses" --> Redis_Command_Abstraction_Implementation + + Connection_Management -- "uses" --> Protocol_Serialization_Deserialization + + Protocol_Serialization_Deserialization -- "uses" --> Utility_Error_Handling + + Redis_Command_Abstraction_Implementation -- "uses" --> Utility_Error_Handling + + Core_Client_API -- "uses" --> Utility_Error_Handling + + Connection_Management -- "uses" --> Utility_Error_Handling + + click Connection_Management href "https://github.com/redis/redis-py/blob/master/.codeboarding//Connection_Management.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The `Protocol & Command Handling` subsystem is a critical part of the `redis-py` client library, responsible for the fundamental communication with the Redis server. + + + +### Protocol Serialization & Deserialization + +This component is responsible for the low-level conversion of Python data structures into the Redis Serialization Protocol (RESP) format for outgoing commands and the efficient parsing of RESP responses received from the Redis server back into native Python data types. It supports different RESP versions (RESP2 and RESP3) and leverages optimized parsers, including the `hiredis` C extension for performance. This component acts as the crucial bridge between the Python application's data representation and Redis's wire protocol. + + + + + +**Related Classes/Methods**: + + + +- `BaseParser` + +- `AsyncBaseParser` + +- `_RESPBase` + +- `_AsyncRESPBase` + +- `_HiredisParser` + +- `_AsyncHiredisParser` + +- `_RESP2Parser` + +- `_AsyncRESP2Parser` + +- `_RESP3Parser` + +- `_AsyncRESP3Parser` + +- `Encoder` + + + + + +### Redis Command Abstraction & Implementation + +This component defines and implements the comprehensive set of Redis commands, providing a Pythonic interface for interacting with Redis functionalities. It encompasses both the core Redis commands (e.g., key-value operations, hashes, lists, sets, sorted sets, streams, geospatial commands, hyperloglog, scanning, ACL, cluster, functions, management, pub/sub, scripting) and specialized commands for various Redis modules (e.g., RedisBloom, RedisJSON, RediSearch, RedisTimeSeries, VectorSearch). This component abstracts the raw command strings and arguments into user-friendly Python methods. + + + + + +**Related Classes/Methods**: + + + +- `CoreCommands` + +- `AsyncCoreCommands` + +- `RedisModuleCommands` + +- `AsyncRedisModuleCommands` + +- `BFCommands` + +- `CFCommands` + +- `JSONCommands` + +- `SearchCommands` + +- `AsyncSearchCommands` + +- `TimeSeriesCommands` + +- `VectorSetCommands` + + + + + +### Core Client API + +Provides a simplified, unified interface to the extensive set of Redis commands. + + + + + +**Related Classes/Methods**: _None_ + + + +### Connection Management [[Expand]](./Connection_Management.md) + +Responsible for the actual sending and receiving of bytes over the network. + + + + + +**Related Classes/Methods**: _None_ + + + +### Utility & Error Handling + +Provides consistent exception handling and potentially shared utility functions. + + + + + +**Related Classes/Methods**: _None_ + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Redis Client Core.md b/.codeboarding/Redis Client Core.md index 7661b5a5e0..3e4dabc7ff 100644 --- a/.codeboarding/Redis Client Core.md +++ b/.codeboarding/Redis Client Core.md @@ -1,120 +1,239 @@ ```mermaid + graph LR + Redis_Client_Core["Redis Client Core"] + Lock_Management["Lock Management"] + Pub_Sub_Component["Pub/Sub Component"] + Pipeline_Transaction_Component["Pipeline/Transaction Component"] + Redis_Client_Core -- "creates instances of" --> Pub_Sub_Component + Redis_Client_Core -- "creates instances of" --> Pipeline_Transaction_Component + Lock_Management -- "utilizes" --> Redis_Client_Core + Pipeline_Transaction_Component -- "extends functionality of" --> Redis_Client_Core + ``` + [![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + ## Component Details + + This graph illustrates the core components of the Redis client library, focusing on the primary interface for Redis interaction, `Redis Client Core`, and its specialized functionalities. It details how `Redis Client Core` orchestrates operations, including the creation of `Pub/Sub Component` for real-time messaging and `Pipeline/Transaction Component` for efficient batched commands. Additionally, it shows how `Lock Management` leverages the `Redis Client Core` for distributed locking, and how the `Pipeline/Transaction Component` builds upon the core client's capabilities. + + ### Redis Client Core + The Redis Client Core component serves as the primary interface for interacting with a Redis server. It manages connection pools, executes commands, handles command parsing, and provides entry points for advanced functionalities like pipelining and publish/subscribe. + + + **Related Classes/Methods**: + + - `redis.client.Redis` (112:670) + - `redis.client.Redis:from_url` (128:176) + - `redis.client.Redis:from_pool` (179:192) + - `redis.client.Redis:__init__` (199:389) + - `redis.client.Redis:get_retry` (405:406) + - `redis.client.Redis:set_retry` (408:410) + - `redis.client.Redis:pipeline` (439:449) + - `redis.client.Redis:transaction` (451:473) + - `redis.client.Redis:pubsub` (556:564) + - `redis.client.Redis:monitor` (566:567) + - `redis.client.Redis:client` (569:572) + - `redis.client.Redis:__exit__` (577:578) + - `redis.client.Redis:__del__` (580:584) + - `redis.client.Redis:_send_command_parse_response` (601:606) + - `redis.client.Redis:execute_command` (622:623) + - `redis.client.Redis:_execute_command` (625:644) + + + ### Lock Management + The Lock Management component provides functionalities for acquiring, releasing, extending, and reacquiring distributed locks in Redis. It handles the underlying Redis commands and manages lock ownership and expiration, raising specific exceptions for lock-related errors. + + + **Related Classes/Methods**: + + - `redis.lock.Lock` (14:343) + - `redis.lock.Lock:__init__` (79:155) + - `redis.lock.Lock:__enter__` (167:173) + - `redis.lock.Lock:__exit__` (175:188) + - `redis.lock.Lock:acquire` (190:235) + - `redis.lock.Lock:release` (265:276) + - `redis.lock.Lock:do_acquire` (237:245) + - `redis.lock.Lock:do_release` (278:285) + - `redis.lock.Lock:extend` (287:302) + - `redis.lock.Lock:do_extend` (304:317) + - `redis.lock.Lock:reacquire` (319:330) + - `redis.lock.Lock:do_reacquire` (332:343) + + + ### Pub/Sub Component + The Pub/Sub Component facilitates real-time messaging through Redis's publish/subscribe mechanism. It allows clients to subscribe to channels or patterns, receive messages, and manage the lifecycle of the pub/sub connection. + + + **Related Classes/Methods**: + + - `redis.client.PubSub` (743:1000) + - `redis.client.PubSub:__init__` (756:791) + - `redis.client.PubSub:__exit__` (796:797) + - `redis.client.PubSub:__del__` (799:806) + - `redis.client.PubSub:close` (823:824) + - `redis.client.PubSub:on_connect` (826:849) + - `redis.client.PubSub:execute_command` (856:880) + - `redis.client.PubSub:clean_health_check_responses` (882:898) + - `redis.client.PubSub:_execute` (910:921) + - `redis.client.PubSub:parse_response` (923:948) + - `redis.client.PubSub:psubscribe` (983:1000) + - `redis.client.PubSub:punsubscribe` (full file reference) + - `redis.client.PubSub:subscribe` (full file reference) + - `redis.client.PubSub:unsubscribe` (full file reference) + - `redis.client.PubSub:ssubscribe` (full file reference) + - `redis.client.PubSub:sunsubscribe` (full file reference) + - `redis.client.PubSub:listen` (full file reference) + - `redis.client.PubSub:get_message` (full file reference) + - `redis.client.PubSub:ping` (full file reference) + - `redis.client.PubSub:handle_message` (full file reference) + - `redis.client.PubSub:run_in_thread` (full file reference) + + + ### Pipeline/Transaction Component + The Pipeline/Transaction Component enables efficient execution of multiple Redis commands by sending them in a single round trip (pipelining) and supports atomic transactions using MULTI/EXEC. It manages command queuing, response parsing, and error handling for batched operations. + + + **Related Classes/Methods**: + + - `redis.client.Pipeline` (1:1000) + - `redis.client.Pipeline:__exit__` (full file reference) + - `redis.client.Pipeline:__del__` (full file reference) + - `redis.client.Pipeline:close` (full file reference) + - `redis.client.Pipeline:multi` (full file reference) + - `redis.client.Pipeline:execute_command` (full file reference) + - `redis.client.Pipeline:_disconnect_reset_raise_on_watching` (full file reference) + - `redis.client.Pipeline:immediate_execute_command` (full file reference) + - `redis.client.Pipeline:_execute_transaction` (full file reference) + - `redis.client.Pipeline:_execute_pipeline` (full file reference) + - `redis.client.Pipeline:raise_first_error` (full file reference) + - `redis.client.Pipeline:annotate_exception` (full file reference) + - `redis.client.Pipeline:parse_response` (full file reference) + - `redis.client.Pipeline:load_scripts` (full file reference) + - `redis.client.Pipeline:_disconnect_raise_on_watching` (full file reference) + - `redis.client.Pipeline:execute` (full file reference) + - `redis.client.Pipeline:discard` (full file reference) + - `redis.client.Pipeline:watch` (full file reference) + - `redis.client.Pipeline:unwatch` (full file reference) + + + + + ### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Utility_Cross_Cutting_Concerns.md b/.codeboarding/Utility_Cross_Cutting_Concerns.md new file mode 100644 index 0000000000..b8e5e314b3 --- /dev/null +++ b/.codeboarding/Utility_Cross_Cutting_Concerns.md @@ -0,0 +1,201 @@ +```mermaid + +graph LR + + Authentication_Security["Authentication & Security"] + + Error_Handling["Error Handling"] + + Retry_Backoff_Strategies["Retry & Backoff Strategies"] + + Event_System["Event System"] + + Caching["Caching"] + + Distributed_Locks["Distributed Locks"] + + General_Utilities["General Utilities"] + + Authentication_Security -- "uses" --> Error_Handling + + Authentication_Security -- "dispatches events to" --> Event_System + + Retry_Backoff_Strategies -- "uses" --> Error_Handling + + Event_System -- "uses" --> Error_Handling + + Caching -- "uses" --> Error_Handling + + Distributed_Locks -- "uses" --> Error_Handling + + General_Utilities -- "uses" --> Error_Handling + + Authentication_Security -- "uses" --> General_Utilities + + Error_Handling -- "uses" --> General_Utilities + + Retry_Backoff_Strategies -- "uses" --> General_Utilities + + Event_System -- "uses" --> General_Utilities + + Caching -- "uses" --> General_Utilities + + Distributed_Locks -- "uses" --> General_Utilities + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +One paragraph explaining the functionality which is represented by this graph. What the main flow is and what is its purpose. + + + +### Authentication & Security + +This component is fundamental as it ensures secure communication and proper authorization, which are paramount for any client library interacting with a data store like Redis. It handles credential provision, token management, and certificate validation (e.g., OCSP stapling), directly addressing the "authentication mechanisms" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/credentials.py` + +- `redis/auth/token.py` + +- `redis/auth/token_manager.py` + +- `redis/ocsp.py` + + + + + +### Error Handling + +This component is crucial for building a robust and user-friendly client library. A well-defined and comprehensive exception hierarchy allows developers to gracefully handle various issues, improving application stability and debugging. It directly addresses the "robust error handling" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/exceptions.py` + + + + + +### Retry & Backoff Strategies + +This component is essential for resilience in distributed systems. Transient failures (e.g., network glitches, temporary server unavailability) are common, and automatic retries with intelligent backoff mechanisms prevent applications from crashing, reduce load on the server during recovery, and improve overall reliability. It directly addresses the "retry strategies" and "backoff" aspects. + + + + + +**Related Classes/Methods**: + + + +- `redis/retry.py` + +- `redis/backoff.py` + + + + + +### Event System + +This component provides a flexible and decoupled way for different parts of the library to communicate and react to internal state changes, such as connection status updates or re-authentication requirements. This promotes a clean architecture and enables dynamic behavior within the client. It directly addresses the "internal event system" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/event.py` + + + + + +### Caching + +This component is fundamental for performance optimization in client libraries. By providing client-side caching capabilities, it reduces redundant network calls to the Redis server, significantly improving response times and reducing server load for frequently accessed data. It directly addresses the "caching" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/cache.py` + + + + + +### Distributed Locks + +This component implements higher-level Redis commands that provide distributed synchronization mechanisms, most notably distributed locks. These primitives are essential for coordinating operations across multiple clients or processes in a distributed environment, preventing race conditions and ensuring data consistency. It directly addresses the "distributed synchronization primitives like locks" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/lock.py` + +- `redis/asyncio/lock.py` + + + + + +### General Utilities + +This component provides a collection of general-purpose helper functions and utilities that are widely used across various parts of the library. It centralizes reusable code, preventing duplication, promoting consistency, and simplifying development across different modules. It directly addresses the "common utilities" aspect. + + + + + +**Related Classes/Methods**: + + + +- `redis/utils.py` + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/on_boarding.md b/.codeboarding/on_boarding.md index 0c78802f5c..58f9912307 100644 --- a/.codeboarding/on_boarding.md +++ b/.codeboarding/on_boarding.md @@ -1,85 +1,181 @@ ```mermaid + graph LR - Redis_Client_Core["Redis Client Core"] - Connection_Protocol_Management["Connection & Protocol Management"] - Command_Module_Execution["Command & Module Execution"] - High_Availability_Cluster_Management["High Availability & Cluster Management"] - Error_Handling["Error Handling"] - Redis_Client_Core -- "uses" --> Connection_Protocol_Management - Redis_Client_Core -- "executes" --> Command_Module_Execution - Redis_Client_Core -- "handles" --> Error_Handling - Connection_Protocol_Management -- "provides to" --> Redis_Client_Core - Connection_Protocol_Management -- "raises" --> Error_Handling - Command_Module_Execution -- "is executed by" --> Redis_Client_Core - Command_Module_Execution -- "uses" --> Connection_Protocol_Management - High_Availability_Cluster_Management -- "extends" --> Redis_Client_Core - High_Availability_Cluster_Management -- "uses" --> Connection_Protocol_Management - Error_Handling -- "is raised by" --> Redis_Client_Core - Error_Handling -- "is raised by" --> Connection_Protocol_Management - click Redis_Client_Core href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Redis Client Core.md" "Details" - click Connection_Protocol_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Connection & Protocol Management.md" "Details" - click Command_Module_Execution href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Command & Module Execution.md" "Details" - click High_Availability_Cluster_Management href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/High Availability & Cluster Management.md" "Details" - click Error_Handling href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main/redis-py/Error Handling.md" "Details" + + Client_API["Client API"] + + Connection_Management["Connection Management"] + + Protocol_Command_Handling["Protocol & Command Handling"] + + Utility_Cross_Cutting_Concerns["Utility & Cross-Cutting Concerns"] + + Client_API -- "interacts with" --> Connection_Management + + Client_API -- "uses" --> Protocol_Command_Handling + + Connection_Management -- "provides connections to" --> Client_API + + Connection_Management -- "leverages" --> Utility_Cross_Cutting_Concerns + + Protocol_Command_Handling -- "used by" --> Client_API + + Protocol_Command_Handling -- "used by" --> Connection_Management + + Utility_Cross_Cutting_Concerns -- "supports" --> Client_API + + Utility_Cross_Cutting_Concerns -- "supports" --> Connection_Management + + click Client_API href "https://github.com/redis/redis-py/blob/master/.codeboarding//Client_API.md" "Details" + + click Connection_Management href "https://github.com/redis/redis-py/blob/master/.codeboarding//Connection_Management.md" "Details" + + click Protocol_Command_Handling href "https://github.com/redis/redis-py/blob/master/.codeboarding//Protocol_Command_Handling.md" "Details" + + click Utility_Cross_Cutting_Concerns href "https://github.com/redis/redis-py/blob/master/.codeboarding//Utility_Cross_Cutting_Concerns.md" "Details" + ``` + + + [![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) -## Component Details -The `redis-py` library provides a Python interface to the Redis key-value store. Its main flow involves establishing connections to Redis servers, executing various Redis commands, handling responses, and managing advanced features like pipelining, transactions, and publish/subscribe. It also supports specialized deployments such as Redis Cluster and Redis Sentinel for high availability and scalability. The architecture is designed to be modular, separating concerns like connection management, command execution, and error handling. -### Redis Client Core -The primary interface for interacting with Redis, encompassing basic command execution, pipelining, transactions, publish/subscribe, and distributed locking mechanisms. It orchestrates high-level operations. +## Details -**Related Classes/Methods**: -- `redis.client.Redis` (112:670) -- `redis.client.Pipeline` (1:1000) -- `redis.client.PubSub` (743:1000) -- `redis.lock.Lock` (14:343) +The redis-py library is designed with a clear layered architecture, where the Client API acts as the primary facade for users. This API delegates connection management to the Connection Management layer, which in turn relies on the Protocol & Command Handling layer for data serialization and deserialization. Various Utility & Cross-Cutting Concerns provide essential supporting services across these layers, ensuring robustness, security, and efficiency. + + + +### Client API [[Expand]](./Client_API.md) + +The primary interface for users to interact with Redis. It provides high-level methods for executing Redis commands, managing command pipelines, and handling transactions. It supports both synchronous and asynchronous operations, abstracting the underlying network communication. + -### Connection & Protocol Management -Manages the lifecycle of connections to Redis servers, including pooling, health checks, and various connection types. It also handles the encoding of commands and parsing of responses, along with authentication and caching. **Related Classes/Methods**: -- `redis.connection.ConnectionPool` (1:1000) -- `redis._parsers.resp3._RESP3Parser` (15:131) -- `redis.auth.token_manager.TokenManager` (121:340) -- `redis.cache.DefaultCache` (142:224) -- `redis.ocsp.OCSPVerifier` (170:308) -### Command & Module Execution -Implements and executes the full range of standard Redis commands (e.g., key-value, list, set, hash, stream operations) and provides interfaces for interacting with various Redis Modules (e.g., JSON, Search, TimeSeries). +- `redis.client` (1:1) + +- `redis.asyncio.client` (1:1) + + + + + +### Connection Management [[Expand]](./Connection_Management.md) + +Responsible for establishing, maintaining, and pooling connections to various Redis deployments (standalone, Sentinel, Cluster). It handles connection lifecycle, including retries, error handling, and routing commands to appropriate nodes based on the deployment type. + + + **Related Classes/Methods**: -- `redis.commands.core.BasicKeyCommands` (1:1000) -- `redis.commands.json.JSON` (1:1000) -### High Availability & Cluster Management -Provides specialized client functionalities for interacting with Redis Cluster and Redis Sentinel setups. It handles node discovery, slot management, command routing in clusters, and master/replica discovery with failover in Sentinel environments. +- `redis.connection` (1:1) + +- `redis.asyncio.connection` (1:1) + +- `redis.sentinel` (1:1) + +- `redis.asyncio.sentinel` (1:1) + +- `redis.cluster` (1:1) + +- `redis.asyncio.cluster` (1:1) + + + + + +### Protocol & Command Handling [[Expand]](./Protocol_Command_Handling.md) + +Encapsulates the logic for converting Python data into the Redis Serialization Protocol (RESP) format for outgoing commands and parsing RESP responses back into Python data types. It also defines the structure and arguments for all Redis commands, including specialized module commands (e.g., RedisJSON, RediSearch). + + + **Related Classes/Methods**: -- `redis.cluster.RedisCluster` (456:1000) -- `redis.sentinel.Sentinel` (198:410) -### Error Handling -Defines and manages custom exception classes for various Redis-related errors, providing a structured and specific way to handle different error scenarios that can occur during client-server interactions. +- `redis._parsers.base` (1:1) + +- `redis._parsers.hiredis` (1:1) + +- `redis._parsers.resp2` (1:1) + +- `redis._parsers.resp3` (1:1) + +- `redis._parsers.encoders` (1:1) + +- `redis.commands.core` (1:1) + +- `redis.commands.bf` (1:1) + +- `redis.commands.json` (1:1) + +- `redis.commands.search` (1:1) + +- `redis.commands.timeseries` (1:1) + +- `redis.commands.vectorset` (1:1) + + + + + +### Utility & Cross-Cutting Concerns [[Expand]](./Utility_Cross_Cutting_Concerns.md) + +A collection of foundational services and common utilities that support the core functionality of the library. This includes authentication mechanisms, robust error handling, retry strategies, an internal event system, caching, and distributed synchronization primitives like locks. + + + **Related Classes/Methods**: -- `redis.exceptions.RedisError` (4:5) + + +- `redis.credentials` (1:1) + +- `redis.auth.token` (1:1) + +- `redis.auth.token_manager` (1:1) + +- `redis.ocsp` (1:1) + +- `redis.exceptions` (1:1) + +- `redis.retry` (1:1) + +- `redis.backoff` (1:1) + +- `redis.event` (1:1) + +- `redis.cache` (1:1) + +- `redis.lock` (1:1) + +- `redis.asyncio.lock` (1:1) + +- `redis.utils` (1:1) + + + + +