redis队列

redis的list支持brpop、blpop,所以能实现队列,fifo、lifo、priority队列可以直接由lpush/rpush和brpop/blpop的组合实现。

  1. fifo和lifo队列,用一个key的list就行了,只做参考。更复杂的功能如get的block、timeout可以照着redis api实现就行了,队列的block需要用到lock,event之类的。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import redis
class FifoRedisQ(object):
def __init__(self, host='localhost', port=6379, db=0, label='test'):
self.r = redis.Redis(host=host, port=port, db=db)
self.label = label
def get(self):
return self.r.brpop(self.label)[-1]
def put(self, item):
self.r.lpush(self.label, item)
class LifoRedisQ(object):
def __init__(self, host='localhost', port=6379, db=0, label='test'):
self.r = redis.Redis(host=host, port=port, db=db)
self.label = label
def get(self):
return self.r.blpop(self.label)[-1]
def put(self, item):
self.r.lpush(self.label, item)
if __name__ == '__main__':
fifoq = FifoRedisQ()
print '----first in first out'
for k in range(0, 10):
fifoq.put(k)
for k in range(0, 10):
print fifoq.get()
print '----last in first out'
lifoq = LifoRedisQ()
for k in range(0, 10):
lifoq.put(k)
for k in range(0, 10):
print lifoq.get()
  1. priority队列,每一个优先级需要用一个key对应一个list,每一个list就是存储该等级的数据channel。有多少种优先级是提前设定好的,不能动态添加。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import redis, random
class PriorityRedisQ(object):
def __init__(self, host='localhost', port=6379, db=0, label='test', weight=[]):
self.r = redis.Redis(host=host, port=port, db=db)
self.label = label
if not weight:
raise Exception("Please set weight of queue.")
self.weight = weight
def get(self):
return self.r.brpop(['_'.join([self.label, str(one)]) for one in self.weight])[-1]
def put(self, priority, item):
self.r.lpush('_'.join([self.label, str(priority)]), (priority, item))
if __name__ == '__main__':
weight = [1, 2, 3, 4]
prq = PriorityRedisQ(weight=weight)
print '----priority'
for k in range(0, 10):
prq.put(random.choice(weight), k)
for k in range(0, 10):
print prq.get()

市面上有那么多好用的队列,你可以不选redis,看具体的业务需要吧,参考webcrawl的redis priority queue