SpringBoot集成Redisson延迟队列

0. 使用场景

1. 下单成功，30分钟未支付。支付超时，自动取消订单
2. 订单签收，签收后7天未进行评价。订单超时未评价，系统默认好评
3. 下单成功，商家5分钟未接单，订单取消
4. 配送超时，推送短信提醒

1.Redisson延迟队列原理

redisson 使用了 两个list + 一个 sorted-set + pub/sub 来实现延时队列，而不是单一的sort-set。

• sorted-set：存放未到期的消息&到期时间，提供消息延时排序功能
• list1：存放未到期消息，作为消息的原始顺序视图，提供如查询、删除指定第几条消息的功能（分析源码得出的，查看哪些地方有使用这个list）
• list2：消费队列，存放到期后的消息，提供消费

整体流程(对应画图PPT链接):

结合源码分析:

• org.redisson.RedissonDelayedQueue#RedissonDelayedQueue
  首先创建延时队列的时候，会创建一个QueueTransferTask, 在里面会订阅一个topic，订阅成功后，执行其pushTask方法，里面会查询sorted-set中100个已到期的消息，将其push到lis2中，并从sorted-set和list1中移除。（这里是为了投递历史未处理的消息）

  protected RedissonDelayedQueue(QueueTransferService queueTransferService, Codec codec, final CommandAsyncExecutor commandExecutor, String name) {
      super(codec, commandExecutor, name);
      channelName = prefixName("redisson_delay_queue_channel", getRawName());
      queueName = prefixName("redisson_delay_queue", getRawName());
      timeoutSetName = prefixName("redisson_delay_queue_timeout", getRawName());
      
      QueueTransferTask task = new QueueTransferTask(commandExecutor.getConnectionManager()) {
          @Override
          protected RFuture<Long> pushTaskAsync() {
              return commandExecutor.evalWriteAsync(getRawName(), LongCodec.INSTANCE, RedisCommands.EVAL_LONG,
                      "local expiredValues = redis.call('zrangebyscore', KEYS[2], 0, ARGV[1], 'limit', 0, ARGV[2]); "
                    + "if #expiredValues > 0 then "
                        + "for i, v in ipairs(expiredValues) do "
                            + "local randomId, value = struct.unpack('dLc0', v);"
                            + "redis.call('rpush', KEYS[1], value);"
                            + "redis.call('lrem', KEYS[3], 1, v);"
                        + "end; "
                        + "redis.call('zrem', KEYS[2], unpack(expiredValues));"
                    + "end; "
                      // get startTime from scheduler queue head task
                    + "local v = redis.call('zrange', KEYS[2], 0, 0, 'WITHSCORES'); "
                    + "if v[1] ~= nil then "
                       + "return v[2]; "
                    + "end "
                    + "return nil;",
                    Arrays.<Object>asList(getRawName(), timeoutSetName, queueName),
                    System.currentTimeMillis(), 100);
          }
          
          @Override
          protected RTopic getTopic() {
              return RedissonTopic.createRaw(LongCodec.INSTANCE, commandExecutor, channelName);
          }
      };
      
      queueTransferService.schedule(queueName, task);
      
      this.queueTransferService = queueTransferService;
  }
  

• org.redisson.RedissonDelayedQueue#offerAsync(V, long, java.util.concurrent.TimeUnit)
  发送延时消息时，会将消息写入 list1和 sorted-set 中，msg会添加一个randomId，支持发送相同的消息。并且判断sorted-set首条消息如果是刚插入的，则publish timeout（到期时间） 到 topic

  public RFuture<Void> offerAsync(V e, long delay, TimeUnit timeUnit) {
      if (delay < 0) {
          throw new IllegalArgumentException("Delay can't be negative");
      }
      
      long delayInMs = timeUnit.toMillis(delay);
      long timeout = System.currentTimeMillis() + delayInMs;
   
      long randomId = ThreadLocalRandom.current().nextLong();
      return commandExecutor.evalWriteAsync(getRawName(), codec, RedisCommands.EVAL_VOID,
              "local value = struct.pack('dLc0', tonumber(ARGV[2]), string.len(ARGV[3]), ARGV[3]);" 
            + "redis.call('zadd', KEYS[2], ARGV[1], value);"
            + "redis.call('rpush', KEYS[3], value);"
            // if new object added to queue head when publish its startTime 
            // to all scheduler workers 
            + "local v = redis.call('zrange', KEYS[2], 0, 0); "
            + "if v[1] == value then "
               + "redis.call('publish', KEYS[4], ARGV[1]); "
            + "end;",
            Arrays.<Object>asList(getRawName(), timeoutSetName, queueName, channelName),
            timeout, randomId, encode(e));
  }

• org.redisson.QueueTransferTask#scheduleTask
  订阅到topic消息后，会先判断其是否临期（delay<10ms），如果是则调用pushTask方法，不是则启动一个定时任务（使用的netty时间轮），延时delay后执行pushTask方法。

  // 订阅topic onMessage 时调用
  private void scheduleTask(final Long startTime) {
      TimeoutTask oldTimeout = lastTimeout.get();
      if (startTime == null) {
          return;
      }
      
      if (oldTimeout != null) {
          oldTimeout.getTask().cancel();
      }
      
      long delay = startTime - System.currentTimeMillis();
      if (delay > 10) {
      // 使用 netty 时间轮 启动一个定时任务
          Timeout timeout = connectionManager.newTimeout(new TimerTask() {                    
              @Override
              public void run(Timeout timeout) throws Exception {
                  pushTask();
                  
                  TimeoutTask currentTimeout = lastTimeout.get();
                  if (currentTimeout.getTask() == timeout) {
                      lastTimeout.compareAndSet(currentTimeout, null);
                  }
              }
          }, delay, TimeUnit.MILLISECONDS);
          if (!lastTimeout.compareAndSet(oldTimeout, new TimeoutTask(startTime, timeout))) {
              timeout.cancel();
          }
      } else {
          pushTask();
      }
  }    
  
  private void pushTask() {
      RFuture<Long> startTimeFuture = pushTaskAsync();
      startTimeFuture.onComplete((res, e) -> {
          if (e != null) {
              if (e instanceof RedissonShutdownException) {
                  return;
              }
              log.error(e.getMessage(), e);
              scheduleTask(System.currentTimeMillis() + 5 * 1000L);
              return;
          }
          
          if (res != null) {
              scheduleTask(res);
          }
      });
  }

2.SpringBoot集成

实验环境:SpringBoot版本3.0.12

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.0.12</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

2.1 引入 Redisson 依赖

 <!--redission-->
<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson-spring-boot-starter</artifactId>
    <version>3.19.0</version>
</dependency>

2.2 配置文件

spring:
  data:
    redis:
      host: 172.19.236.66
      port: 6379
      #password: 123456
      database: 0
      timeout: 3000

2.3 创建 RedissonConfig 配置

package com.example.redissionstudy.config;

import org.redisson.Redisson;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * @author LuoJia
 * @version 1.0
 * @description: Redis链接配置文件
 * @date 2023/11/3 8:59
 */
@Configuration
public class RedissonConfig {

    @Value("${spring.data.redis.host}")
    private String host;
    @Value("${spring.data.redis.port}")
    private int port;
    @Value("${spring.data.redis.database}")
    private int database;
    //@Value("${spring.data.redis.password}")
    //private String password;

    @Bean
    public RedissonClient redissonClient() {
        Config config = new Config();
        config.useSingleServer()
                .setAddress("redis://" + host + ":" + port)
                .setDatabase(database);
                //.setPassword(password)
        return Redisson.create(config);
    }
}

• 测试使用

@SpringBootTest
@Slf4j
class RedissionStudyApplicationTests {
    @Resource
    RedissonClient redissonClient;

    @Test
    void testRedission() {
        //字符串操作
        RBucket<String> rBucket =  redissonClient.getBucket("strKey");
        // 设置value和key的有效期
        rBucket.set("张三", 30, TimeUnit.MINUTES);
        // 通过key获取value
        System.out.println(redissonClient.getBucket("strKey").get());
    }
}

张三

• redis查看结果

127.0.0.1:6379> keys str*
1) "strKey"
127.0.0.1:6379> get strKey
"\x03\x83\xe5\xbc\xa0\xe4\xb8\x89"

2.4 封装 Redis 延迟队列工具类

package com.example.redissionstudy.utils;

import lombok.extern.slf4j.Slf4j;
import org.redisson.api.RBlockingDeque;
import org.redisson.api.RDelayedQueue;
import org.redisson.api.RedissonClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.util.Map;
import java.util.concurrent.TimeUnit;

/**
 * @author LuoJia
 * @version 1.0
 * @description:  Redission 延迟队列工具类
 * @date 2023/11/3 9:51
 */
@Slf4j
@Component
public class RedisDelayQueueUtil {

    @Autowired
    private RedissonClient redissonClient;

    /**
     * 添加延迟队列
     *
     * @param value     队列值
     * @param delay     延迟时间
     * @param timeUnit  时间单位
     * @param queueCode 队列键
     * @param <T>
     */
    public <T> void addDelayQueue(T value, long delay, TimeUnit timeUnit, String queueCode) {
        try {
            RBlockingDeque<Object> blockingDeque = redissonClient.getBlockingDeque(queueCode);
            RDelayedQueue<Object> delayedQueue = redissonClient.getDelayedQueue(blockingDeque);
            delayedQueue.offer(value, delay, timeUnit);
            log.info("(添加延时队列成功) 队列键：{}，队列值：{}，延迟时间：{}", queueCode, value, timeUnit.toSeconds(delay) + "秒");
        } catch (Exception e) {
            log.error("(添加延时队列失败) {}", e.getMessage());
            throw new RuntimeException("(添加延时队列失败)");
        }
    }

    /**
     * 获取延迟队列
     *
     * @param queueCode
     * @param <T>
     * @return
     * @throws InterruptedException
     */
    public <T> T getDelayQueue(String queueCode) throws InterruptedException {
        RBlockingDeque<Map> blockingDeque = redissonClient.getBlockingDeque(queueCode);
        T value = (T) blockingDeque.take();
        return value;
    }
}

2.5 创建延迟队列业务枚举

package com.example.redissionstudy.enums;

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;

/**
 * @author LuoJia
 * @version 1.0
 * @description: 延迟队列业务枚举
 * @date 2023/11/3 9:53
 */
@Getter
@AllArgsConstructor
@NoArgsConstructor
public enum RedisDelayQueueEnum {
    ORDER_PAYMENT_TIMEOUT("ORDER_PAYMENT_TIMEOUT", "订单支付超时，自动取消订单", "orderPaymentTimeout"),
    ORDER_TIMEOUT_NOT_EVALUATED("ORDER_TIMEOUT_NOT_EVALUATED", "订单超时未评价，系统默认好评", "orderTimeoutNotEvaluated");

    /**
     * 延迟队列 RedisKey
     */
    private String code;

    /**
     * 中文描述
     */
    private String name;

    /**
     * 延迟队列具体业务实现的 Bean
     * 可通过 Spring 的上下文获取
     */
    private String beanId;
}

2.6 定义延迟队列执行器

package com.example.redissionstudy.handler;

/**
 * @author LuoJia
 * @version 1.0
 * @description: 延迟队列执行器接口
 * @date 2023/11/3 9:58
 */
public interface RedisDelayQueueHandle<T>{
    void execute(T t);
}

2.7 创建枚举中定义的Bean，并实现延迟队列执行器

• OrderPaymentTimeout：订单支付超时延迟队列处理类

package com.example.redissionstudy.handler.impl;

import com.example.redissionstudy.enums.RedisDelayQueueEnum;
import com.example.redissionstudy.handler.RedisDelayQueueHandle;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;

import java.util.Map;

/**
 * @author LuoJia
 * @version 1.0
 * @description: 订单支付超时处理类
 * @date 2023/11/3 10:00
 */
@Component
@Slf4j
public class OrderPaymentTimeout implements RedisDelayQueueHandle<Map> {
    @Override
    public void execute(Map map) {
        log.info("{} {}", RedisDelayQueueEnum.ORDER_PAYMENT_TIMEOUT.getName(), map);
        // TODO 订单支付超时，自动取消订单处理业务...
    }
}

• OrderTimeoutNotEvaluated：订单超时未评价延迟队列处理类

package com.example.redissionstudy.handler.impl;

import com.example.redissionstudy.enums.RedisDelayQueueEnum;
import com.example.redissionstudy.handler.RedisDelayQueueHandle;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;

import java.util.Map;

/**
 * @author LuoJia
 * @version 1.0
 * @description: 订单超时未评价处理类
 * @date 2023/11/3 10:01
 */
@Component
@Slf4j
public class OrderTimeoutNotEvaluated implements RedisDelayQueueHandle<Map> {
    @Override
    public void execute(Map map) {
        log.info("{} {}", RedisDelayQueueEnum.ORDER_TIMEOUT_NOT_EVALUATED.getName(), map);
        // TODO 订单超时未评价，系统默认好评处理业务...
    }
}

2.8 创建延迟队列消费线程，项目启动完成后开启

package listener;

import com.example.redissionstudy.enums.RedisDelayQueueEnum;
import com.example.redissionstudy.handler.RedisDelayQueueHandle;
import com.example.redissionstudy.utils.RedisDelayQueueUtil;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.context.ApplicationContext;
import org.springframework.stereotype.Component;

/**
 * @author LuoJia
 * @version 1.0
 * @description: 启动延迟队列
 * @date 2023/11/3 10:02
 */
@Slf4j
@Component
public class RedisDelayQueueRunner implements CommandLineRunner {
    @Autowired
    private RedisDelayQueueUtil redisDelayQueueUtil;
    @Autowired
    private ApplicationContext applicationContext;
    @Override
    public void run(String... args) {
        new Thread(() -> {
            while (true) {
                try {
                    RedisDelayQueueEnum[] queueEnums = RedisDelayQueueEnum.values();
                    for (RedisDelayQueueEnum queueEnum : queueEnums) {
                        Object value = redisDelayQueueUtil.getDelayQueue(queueEnum.getCode());
                        if (value != null) {
                            RedisDelayQueueHandle redisDelayQueueHandle = (RedisDelayQueueHandle) applicationContext.getBean(queueEnum.getBeanId());
                            redisDelayQueueHandle.execute(value);
                        }
                    }
                } catch (InterruptedException e) {
                    log.error("(Redis延迟队列异常中断) {}", e.getMessage());
                }
            }
        }).start();
        log.info("(Redis延迟队列启动成功)");
    }
}

以上步骤，Redis 延迟队列核心代码已经完成，下面我们写一个测试接口，用 PostMan 模拟测试一下

2.9 创建一个测试接口，模拟添加延迟队列

package com.example.redissionstudy.controller;

/**
 * @author LuoJia
 * @version 1.0
 * @description: 延迟队列测试
 * @date 2023/11/3 10:05
 */

import com.example.redissionstudy.enums.RedisDelayQueueEnum;
import com.example.redissionstudy.utils.RedisDelayQueueUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;

@RestController
public class RedisDelayQueueController {

    @Autowired
    private RedisDelayQueueUtil redisDelayQueueUtil;

    @GetMapping("/addQueue")
    public void addQueue() {
        Map<String, String> map1 = new HashMap<>();
        map1.put("orderId", "100");
        map1.put("remark", "其他信息");

        Map<String, String> map2 = new HashMap<>();
        map2.put("orderId", "200");
        map2.put("remark", "其他信息");

        // 添加订单支付超时，自动取消订单延迟队列。为了测试效果，延迟10秒钟
        redisDelayQueueUtil.addDelayQueue(map1, 10, TimeUnit.SECONDS, RedisDelayQueueEnum.ORDER_PAYMENT_TIMEOUT.getCode());

        // 订单超时未评价，系统默认好评。为了测试效果，延迟20秒钟
        redisDelayQueueUtil.addDelayQueue(map2, 20, TimeUnit.SECONDS, RedisDelayQueueEnum.ORDER_TIMEOUT_NOT_EVALUATED.getCode());
    }
}

• 运行结果

2023-11-03T10:09:46.800+08:00  INFO 21480 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2023-11-03T10:09:46.816+08:00  INFO 21480 --- [           main] c.e.r.RedissionStudyApplication          : Started RedissionStudyApplication in 4.888 seconds (process running for 5.743)
2023-11-03T10:09:46.825+08:00  INFO 21480 --- [           main] c.e.r.listener.RedisDelayQueueRunner     : (Redis延迟队列启动成功)
2023-11-03T10:09:47.039+08:00  INFO 21480 --- [-10.108.155.252] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-11-03T10:09:47.040+08:00  INFO 21480 --- [-10.108.155.252] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2023-11-03T10:09:47.042+08:00  INFO 21480 --- [-10.108.155.252] o.s.web.servlet.DispatcherServlet        : Completed initialization in 2 ms
2023-11-03T10:10:25.798+08:00  INFO 21480 --- [nio-8080-exec-4] c.e.r.utils.RedisDelayQueueUtil          : (添加延时队列成功) 队列键：ORDER_PAYMENT_TIMEOUT，队列值：{orderId=100, remark=其他信息}，延迟时间：10秒
2023-11-03T10:10:25.802+08:00  INFO 21480 --- [nio-8080-exec-4] c.e.r.utils.RedisDelayQueueUtil          : (添加延时队列成功) 队列键：ORDER_TIMEOUT_NOT_EVALUATED，队列值：{orderId=200, remark=其他信息}，延迟时间：20秒
2023-11-03T10:10:35.779+08:00  INFO 21480 --- [       Thread-2] c.e.r.handler.impl.OrderPaymentTimeout   : 订单支付超时，自动取消订单 {orderId=100, remark=其他信息}
2023-11-03T10:10:45.860+08:00  INFO 21480 --- [       Thread-2] c.e.r.h.impl.OrderTimeoutNotEvaluated    : 订单超时未评价，系统默认好评 {orderId=200, remark=其他信息}

参考资料

1. SpringBoot集成Redisson实现延迟队列 - 掘金 (juejin.cn)
2. SpringBoot集成Redisson实现延迟队列_redssion延时队列订阅_刘鹏博.的博客-CSDN博客
3. Maven Repository: org.redisson » redisson-spring-boot-starter (mvnrepository.com)
4. 【进阶篇】Redis实战之Redisson使用技巧详解 - 知乎 (zhihu.com)
5. Table of Content · redisson/redisson Wiki · GitHub
6. 浅析 Redisson 的分布式延时队列 RedissonDelayedQueue 运行流程 - 掘金 (juejin.cn)
7. Redisson分布式延时队列 RedissonDelayedQueue - 掘金 (juejin.cn)