The goal of this script is to create a pub/sub pattern that helps users to subscribe to publisher channels and get notified whenever there are updates. First, we will create a python script with the following code:
I am so surprised the redis library for python isn't using these defaults as fallbacks to environment variables if None is received for host, port, password (allowing for override, but also avoiding cluttering code) -py/issues/970
python redis-py
A couple weekends ago I got it into my head that I would build a thin Python wrapper for working with Redis. Andy McCurdy's redis-py is a fantastic low-level client library with built-in support for connection-pooling and pipelining, but it does little more than provide an interface to Redis' built-in commands (and rightly so). I decided to build a project on top of redis-py that exposed pythonic containers for the Redis data-types. I went on to add a few extras, including a cache and a declarative model layer. The result is walrus.
Redis supports five data-types, and each of these types supports a number of special-purpose commands. To make working with these types easier, I wrote container objects that look like their built-in analogues. For instance walrus hashes look like dict objects, have familiar methods like keys(), items(), update(), and support item access using square-brackets. Walrus sets behave like python sets, and so on.
Redis is a powerful, lightning-fast key-value store. Gevent (1.0) is an event-driven concurrency framework for Python (based on greenlet and libev) allowing for highly concurrent networking and tackling the C10K problem. Redis-py, the well-established Redis client for Python, makes Python talk to Redis. The communication takes place through sockets and is request-response-based. A typical Redis-based Python application therefore is I/O-bound rather than CPU-bound. Furthermore, various features of Redis implicate that a request is not immediately followed by a response, making certain requests block the calling thread for an arbitrary amount of time. This is where gevent comes into play: it allows for concurrent execution of these blocking requests within coroutines. In this blog post, I am presenting a short code example integrating gevent with redis-py.
We have a Python web service where we store some key-val pairs in redis.Occasionally, I want to delete some of the keys matching a certain pattern.Current, we are using redis-py-cluster for redis-related operations.
An efficient and user-friendly async redis client ported from redis-py(which is a Python interface to the Redis key-value). And the cluster part is ported from redis-py-clusteraredis is the async version of these to redis clients, with effort to enable you using redis with asyncio more easily.
The following article shows how to use redis to build a simplemulti-producer, multi-consumer Queue with an interface similar to the pythonstandardlib queue. With this queue you can easily share databetween multiple processes or offload time consumig calculations to multiple workerprocesses.
The official Redis documentation does a great job of explaining each commandin detail. redis-py exposes two client classesthat implement these commands. The StrictRedis class attempts to adhere to theofficial official command syntax.
As a result, redis-py does not implement the SELECT command on client instances.If you use multiple Redis databases within the same application, you shouldcreate a separate client instance (and possibly a separate connection pool) foreach database.
Python: set_get.py 1 2 3 4 5 6 7 8 910111213141516#!/usr/bin/env python# set_get.pyimport redisR = redis.Redis(host='localhost',port=12987,decode_responses=True)try: R.ping()except redis.exceptions.ConnectionError as e: print(f'Is Redis running? Unable to connect e') sys.exit(1)print(f'Connected to server.')R.set('sensor_T712_temperature', 39.8)retrieved_temp = R.get('sensor_T712_temperature')print(f'sensor_T712_temperature = retrieved_temp')print(f'(as a number) = float(retrieved_temp):.3f')
Python: get_hash.py 1 2 3 4 5 6 7 8 91011121314151617#!/usr/bin/env python# get_hash.pyimport redisR = redis.Redis(host='localhost',port=12987,decode_responses=True)try: R.ping()except redis.exceptions.ConnectionError as e: print(f'Is Redis running? Unable to connect e') sys.exit(1)print(f'Connected to server.')values = R.hmget('newmark', ['alpha', 'beta', 'gamma'])newmark = 'alpha' : float(values[0]), 'beta' : float(values[1]), 'gamma' : values[2] print(newmark)
Python: set_get_speed.py 1 2 3 4 5 6 7 8 9101112131415161718192021222324252627#!/usr/bin/env python# set_get_speed.pyimport redisimport timeR = redis.Redis(host='localhost',port=12987)try: R.ping()except redis.exceptions.ConnectionError as e: print(f'Is Redis running? Unable to connect e') sys.exit(1)print(f'Connected to server.')N = 100_000T_s = time.time()for i in range(N): R.set(f'K_%06i', i)T_e = time.time()print(f'N sets for integer values took T_e-T_s:.3f s')T_s = time.time()for i in range(N): v = R.get(f'K_%06i') v = v.decode()T_e = time.time()print(f'N gets for integer values took T_e-T_s:.3f s')
Python: subscribe.py 1 2 3 4 5 6 7 8 9101112131415161718192021222324252627282930313233343536#!/usr/bin/env python# subscribe.pyimport timeimport sysimport redisR = redis.Redis(host='localhost',port=12987,decode_responses=True)try: R.ping()except redis.exceptions.ConnectionError as e: print(f'Is Redis running? Unable to connect e') sys.exit(1)print(f'Connected to server.')# modify configuration to enable keyspace notificationR.config_set('notify-keyspace-events', 'KEA')Sub = R.pubsub()keys_to_match = 'sensor_*_temperature'Sub.psubscribe('__keyspace@0__:' + keys_to_match)while True: try: message = Sub.get_message() except redis.exceptions.ConnectionError: print('lost connection to server') sys.exit(1) if message is None: time.sleep(0.1) continue keyname = message['channel'].replace('__keyspace@0__:','') if keyname == keys_to_match: # initial subscription value continue # gets here if any of the matching keys was updated value = R.get(keyname) print(f'keyname = value')
The purpose of walrus is to makeworking with Redis in Python a little easier. Rather than ask you to learn anew library, walrus subclasses and extends the popular redis-py client,allowing it to be used as a drop-in replacement. In addition to all thefeatures in redis-py, walrus adds support for some newer commands,including full support for streams and consumer groups. 2ff7e9595c
Comments