-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsearch.xml
111 lines (53 loc) · 60.8 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>地表最详细MySQL行转列</title>
<link href="/article/2023/20646/"/>
<url>/article/2023/20646/</url>
<content type="html"><![CDATA[<h1 id="地表最详细MySQL行转列"><a href="#地表最详细MySQL行转列" class="headerlink" title="地表最详细MySQL行转列"></a>地表最详细MySQL行转列</h1><p>业务需求需要将存储的数据实现行转列展示，所以弄了个案例来分析一波行转列，走起⇒</p><h2 id="业务背景"><a href="#业务背景" class="headerlink" title="业务背景"></a>业务背景</h2><p>学校里面有记录每个学生的每一科成绩，每个学生选得课又不全相同，所以产生了学生科目成绩分析表。</p><p><strong>原始数据效果图：</strong></p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow2.png" alt="效果图"></p><p><strong>行转列后效果图:</strong></p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow1.png" alt="效果图"></p><h2 id="数据表结构"><a href="#数据表结构" class="headerlink" title="数据表结构"></a>数据表结构</h2><ul><li><p>学生表</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">CREATE TABLE `student` ( `stuid` VARCHAR(16) NOT NULL COMMENT '学号', `stunm` VARCHAR(20) NOT NULL COMMENT '学生姓名', PRIMARY KEY (`stuid`))COLLATE='utf8_general_ci'ENGINE=InnoDB;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre></li><li><p>课程表</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">CREATE TABLE `courses` ( `courseno` VARCHAR(20) NOT NULL, `coursenm` VARCHAR(100) NOT NULL, PRIMARY KEY (`courseno`))COMMENT='课程表'COLLATE='utf8_general_ci'ENGINE=InnoDB;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre></li><li><p>成绩表</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">CREATE TABLE `score` ( `stuid` VARCHAR(16) NOT NULL, `courseno` VARCHAR(20) NOT NULL, `scores` FLOAT NULL DEFAULT NULL, PRIMARY KEY (`stuid`, `courseno`))COLLATE='utf8_general_ci'ENGINE=InnoDB;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre></li><li><p>原始数据效果图的Sql</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">select st.stuid, st.stunm, c.coursenm, s.scoresFrom student stLeft Join score s On st.stuid = s.stuidLeft Join courses c On c.courseno = s.courseno;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span></span></code></pre><p>从这<font color="#FF0000"> Sql</font> 中能清晰看出各表的关系。</p><p><font color="#FF0000"> <strong>知识点:</strong></font>MyISAM与InnoDB 的区别（9个不同点）</p></li></ul><p><strong>InnoDB和MyISAM是很多人在使用MySQL时最常用的两个表类型，这两个表类型各有优劣，5.7之后就不一样了。</strong> </p><p> 1. InnoDB支持事务，MyISAM不支持，对于InnoDB每一条SQL语言都默认封装成事务，自动提交，这样会影响速度，所以最好把多条SQL语言放在begin和 commit之间，组成一个事务；</p><p>2. InnoDB支持外键，而MyISAM不支持。对一个包含外键的InnoDB表转为MYISAM会失败； </p><p>3. InnoDB是聚集索引，使用B+Tree作为索引结构，数据文件是和（主键）索引绑在一起的（表数据文件本身就是按B+Tree组织的一个索引结构），必须要有主键，通过主键索引效率很高。但是辅助索引需要两次查询，先查询到主键，然后再通过主键查询到数据。因此，主键不应该过大，因为主键太大，其他索引也都会很大。</p><p>MyISAM是非聚集索引，也是使用B+Tree作为索引结构，索引和数据文件是分离的，索引保存的是数据文件的指针。主键索引和辅助索引是独立的。</p><p> 也就是说：InnoDB的B+树主键索引的叶子节点就是数据文件，辅助索引的叶子节点是主键的值；而MyISAM的B+树主键索引和辅助索引的叶子节点都是数据文件的地址指针。</p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow3.png" alt="效果图"></p><p>4. InnoDB 中不保存表的具体行数，也就是说，执行select count() from table时，InnoDB要扫描一遍整个表来计算有多少行，但是MyISAM只要简单的 读出保存好的行数即可。注意的是，当count()语句包含 where条件时，两种表的操作是一样的。</p><p>5. Innodb不支持全文索引，而MyISAM支持全文索引，在涉及全文索引领域的查询效率上MyISAM速度更快高；</p><p>PS：<font color="#FF0000"> 5.7以后的InnoDB支持全文索引了</font></p><p>6. MyISAM表格可以被压缩后进行查询操作</p><p>7. InnoDB支持表、行(默认)级锁，而MyISAM支持表级锁</p><p>8. InnoDB表必须有唯一索引（如主键）（用户没有指定的话会自己找/生产一个隐藏列Row_id来充当默认主键），而Myisam可以没有</p><p>9.Innodb存储文件有frm、ibd，而Myisam是frm、MYD、MYI<br> Innodb：frm是表定义文件，ibd是数据文件<br> Myisam：frm是表定义文件，myd是数据文件，myi是索引文件</p><p><font color="#FF0000"> 两种类型最主要的差别就是Innodb 支持事务处理与外键和行级锁.而MyISAM不支持。</font></p><p><font color="#FF0000"><strong>如何选择：</strong></font></p><p>1.是否要支持事务，如果要请选择innodb，如果不需要可以考虑MyISAM；<br> 2.如果表中绝大多数都只是读查询，可以考虑MyISAM，如果既有读也有写，请使用InnoDB。<br> 3.系统奔溃后，MyISAM恢复起来更困难，能否接受；<br> 4.MySQL5.5版本开始Innodb已经成为Mysql的默认引擎(之前是MyISAM)，说明其优势是有目共睹的，如果你不知道用什么，那就用InnoDB，至少不会差。</p><h2 id="模拟数据准备"><a href="#模拟数据准备" class="headerlink" title="模拟数据准备"></a>模拟数据准备</h2><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">/*学生表数据*/Insert Into student (stuid, stunm) Values('1001', '张三');Insert Into student (stuid, stunm) Values('1002', '李四');Insert Into student (stuid, stunm) Values('1003', '赵二');Insert Into student (stuid, stunm) Values('1004', '王五');Insert Into student (stuid, stunm) Values('1005', '刘青');Insert Into student (stuid, stunm) Values('1006', '周明');<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">/*课程表数据*/Insert Into courses (courseno, coursenm) Values('C001', '大学语文');Insert Into courses (courseno, coursenm) Values('C002', '新视野英语');Insert Into courses (courseno, coursenm) Values('C003', '离散数学');Insert Into courses (courseno, coursenm) Values('C004', '概率论与数理统计');Insert Into courses (courseno, coursenm) Values('C005', '线性代数');Insert Into courses (courseno, coursenm) Values('C006', '高等数学(一)');Insert Into courses (courseno, coursenm) Values('C007', '道德与法治');<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">/*成绩表数据*/Insert Into score(stuid, courseno, scores) Values('1001', 'C001', 67);Insert Into score(stuid, courseno, scores) Values('1002', 'C001', 68);Insert Into score(stuid, courseno, scores) Values('1003', 'C001', 69);Insert Into score(stuid, courseno, scores) Values('1004', 'C001', 70);Insert Into score(stuid, courseno, scores) Values('1005', 'C001', 71);Insert Into score(stuid, courseno, scores) Values('1006', 'C001', 72);Insert Into score(stuid, courseno, scores) Values('1001', 'C002', 87);Insert Into score(stuid, courseno, scores) Values('1002', 'C002', 88);Insert Into score(stuid, courseno, scores) Values('1003', 'C002', 89);Insert Into score(stuid, courseno, scores) Values('1004', 'C002', 90);Insert Into score(stuid, courseno, scores) Values('1005', 'C002', 91);Insert Into score(stuid, courseno, scores) Values('1006', 'C002', 92);Insert Into score(stuid, courseno, scores) Values('1001', 'C003', 83);Insert Into score(stuid, courseno, scores) Values('1002', 'C003', 84);Insert Into score(stuid, courseno, scores) Values('1003', 'C003', 85);Insert Into score(stuid, courseno, scores) Values('1004', 'C003', 86);Insert Into score(stuid, courseno, scores) Values('1005', 'C003', 87);Insert Into score(stuid, courseno, scores) Values('1006', 'C003', 88);Insert Into score(stuid, courseno, scores) Values('1001', 'C004', 88);Insert Into score(stuid, courseno, scores) Values('1002', 'C004', 89);Insert Into score(stuid, courseno, scores) Values('1003', 'C004', 90);Insert Into score(stuid, courseno, scores) Values('1004', 'C004', 91);Insert Into score(stuid, courseno, scores) Values('1005', 'C004', 92);Insert Into score(stuid, courseno, scores) Values('1006', 'C004', 93);Insert Into score(stuid, courseno, scores) Values('1001', 'C005', 77);Insert Into score(stuid, courseno, scores) Values('1002', 'C005', 78);Insert Into score(stuid, courseno, scores) Values('1003', 'C005', 79);Insert Into score(stuid, courseno, scores) Values('1004', 'C005', 80);Insert Into score(stuid, courseno, scores) Values('1005', 'C005', 81);Insert Into score(stuid, courseno, scores) Values('1006', 'C005', 82);Insert Into score(stuid, courseno, scores) Values('1001', 'C006', 77);Insert Into score(stuid, courseno, scores) Values('1002', 'C006', 78);Insert Into score(stuid, courseno, scores) Values('1003', 'C006', 79);Insert Into score(stuid, courseno, scores) Values('1004', 'C006', 80);Insert Into score(stuid, courseno, scores) Values('1005', 'C006', 81);Insert Into score(stuid, courseno, scores) Values('1006', 'C006', 82);<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><hr><h2 id="静态行转列实现"><a href="#静态行转列实现" class="headerlink" title="静态行转列实现"></a>静态行转列实现</h2><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">Select st.stuid, st.stunm, MAX(CASE c.coursenm WHEN '大学语文' THEN s.scores ELSE 0 END ) '大学语文', MAX(CASE c.coursenm WHEN '新视野英语' THEN ifnull(s.scores,0) ELSE 0 END ) '新视野英语', MAX(CASE c.coursenm WHEN '离散数学' THEN ifnull(s.scores,0) ELSE 0 END ) '离散数学', MAX(CASE c.coursenm WHEN '概率论与数理统计' THEN ifnull(s.scores,0) ELSE 0 END ) '概率论与数理统计', MAX(CASE c.coursenm WHEN '线性代数' THEN ifnull(s.scores,0) ELSE 0 END ) '线性代数', MAX(CASE c.coursenm WHEN '高等数学(一)' THEN ifnull(s.scores,0) ELSE 0 END ) '高等数学(一)', MAX(CASE c.coursenm WHEN '道德与法治' THEN ifnull(s.scores,0) ELSE 0 END ) '道德与法治'From student stLeft Join score s On st.stuid = s.stuidLeft Join courses c On c.courseno = s.coursenoGroup by st.stuid;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>查询结果如图:</p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow4.png" alt="效果图"></p><p>从上面语句可以看出，知道固定的列，可以使用语句来实现：</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">MAX(CASE c.coursenm WHEN '大学语文' THEN s.scores ELSE 0 END ) '大学语文'<span aria-hidden="true" class="line-numbers-rows"><span></span></span></code></pre><p><font color="#20B2AA">或者</font></p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">SUM(CASE c.coursenm WHEN '大学语文' THEN s.scores ELSE 0 END ) '大学语文'<span aria-hidden="true" class="line-numbers-rows"><span></span></span></code></pre><p>从这语句可以慢慢推出实现动态行转列的方法了，感觉来啦⇓</p><h2 id="动态行转列"><a href="#动态行转列" class="headerlink" title="动态行转列"></a>动态行转列</h2><p>脑子飞速转动，该怎么实现呢？？？</p><ul><li><p>首先要获取下面的语句</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">MAX(CASE c.coursenm WHEN '大学语文' THEN s.scores ELSE 0 END ) '大学语文',MAX(CASE c.coursenm WHEN '新视野英语' THEN ifnull(s.scores,0) ELSE 0 END ) '新视野英语', MAX(CASE c.coursenm WHEN '离散数学' THEN ifnull(s.scores,0) ELSE 0 END ) '离散数学',<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span></span></code></pre></li><li><p>具体获取方式，见下面SQL</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">SELECT GROUP_CONCAT(DISTINCT CONCAT( 'MAX(IF(c.coursenm = ''', c.coursenm, ''', s.scores, 0)) AS ''', c.coursenm, '''' ) )FROM courses c;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>得到的结果为：</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">MAX(IF(c.coursenm = '大学语文', s.scores, 0)) AS '大学语文',MAX(IF(c.coursenm = '新视野英语', s.scores, 0)) AS '新视野英语',MAX(IF(c.coursenm = '概率论与数理统计', s.scores, 0)) AS '概率论与数理统计',MAX(IF(c.coursenm = '离散数学', s.scores, 0)) AS '离散数学',MAX(IF(c.coursenm = '线性代数', s.scores, 0)) AS '线性代数',MAX(IF(c.coursenm = '道德与法治', s.scores, 0)) AS '道德与法治',MAX(IF(c.coursenm = '高等数学(一)', s.scores, 0)) AS '高等数学(一)'<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre></li><li><p>动态列已经得到，在结合查询SQL得到效果会是咋样呢？？</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">Select st.stuid, st.stunm, ( SELECT GROUP_CONCAT(DISTINCT CONCAT( 'MAX(IF(c.coursenm = ''', c.coursenm, ''', s.scores, NULL)) AS ', c.coursenm ) ) FROM courses c)From student stLeft Join score s On st.stuid = s.stuidLeft Join courses c On c.courseno = s.coursenoGroup by st.stuid;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>等到的查询结果居然还不能满足呢，如下图：</p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow5.png" alt="效果图"></p></li><li><p>像普通的语句查询是无法实现的啦，只能使出语句拼接的绝招啦</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">SET @sql = NULL;SELECT GROUP_CONCAT(DISTINCT CONCAT( 'MAX(IF(c.coursenm = ''', c.coursenm, ''', s.scores, 0)) AS ''', c.coursenm, '''' ) ) INTO @sqlFROM courses c; SET @sql = CONCAT('Select st.stuid, st.stunm, ', @sql, ' From student st Left Join score s On st.stuid = s.stuid Left Join courses c On c.courseno = s.courseno Group by st.stuid');PREPARE stmt FROM @sql;EXECUTE stmt;DEALLOCATE PREPARE stmt;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>不容易呀，终于得到结果：</p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow4.png" alt="效果图"></p><p><font color="#FF0000">MySQL prepare语法： </font></p><p><font color="#FF0000">PREPARE statement_name FROM preparable_SQL_statement; /<em>定义</em>/ </font></p><p><font color="#FF0000">EXECUTE statement_name [USING @var_name [, @var_name] …]; /<em>执行预处理语句</em>/ </font><br><font color="#FF0000">{DEALLOCATE | DROP} PREPAREstatement_name /<em>删除定义</em>/ ; </font></p></li><li><p>需求又来了，假如需要查询学号为1005的成绩，效果如图：</p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow6.png" alt="效果图"></p></li><li><p>该如何实现呢？？下面就是见证奇迹的SQL</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">SET @sql = NULL;SET @stuid = '1005';SELECT GROUP_CONCAT(DISTINCT CONCAT( 'MAX(IF(c.coursenm = ''', c.coursenm, ''', s.scores, 0)) AS ''', c.coursenm, '''' ) ) INTO @sqlFROM courses c; SET @sql = CONCAT('Select st.stuid, st.stunm, ', @sql, ' From Student st Left Join score s On st.stuid = s.stuid Left Join courses c On c.courseno = s.courseno Where st.stuid = ''', @stuid, ''' Group by st.stuid');PREPARE stmt FROM @sql;EXECUTE stmt;DEALLOCATE PREPARE stmt;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre></li><li><p>直接在<a href="http://lib.csdn.net/base/mysql">MySQL</a>中操作是没问题的，需求完成了，不过又进到另外一个坑，项目是使用MyBatis，大家都知道在MyBatis中的XML文件中可以自己写SQL语句，但是这样的很显然我们没法放到XML文件中。</p><p>而且最关键的是，这里不能用 If 条件，好比我们要判断学号是否为空或者等于0再加上条件进行查询，可是这里不支持。</p><p>没错就是下面这样</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">SET @sql = NULL;SET @stuid = '1003';SET @courseno = 'C002'; SELECT GROUP_CONCAT(DISTINCT CONCAT( 'MAX(IF(c.coursenm = ''', c.coursenm, ''', s.scores, 0)) AS ''', c.coursenm, '''' ) ) INTO @sqlFROM courses c; SET @sql = CONCAT('Select st.stuid, st.stunm, ', @sql, ' From student st Left Join score s On st.stuid = s.stuid Left Join courses c On c.courseno = s.courseno'); IF @stuid is not null and @stuid != 0 thenSET @sql = CONCAT(@sql, ' Where st.stuid = ''', @stuid, '''');END IF; SET @sql = CONCAT(@sql, ' Group by st.stuid'); PREPARE stmt FROM @sql;EXECUTE stmt;DEALLOCATE PREPARE stmt;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>对，我就是加上 if 之后人家就是不支持，就是这么任性。</p></li><li><p>只能使用存储过程啦，而且又方便项目调用，不啰嗦啦，直接上Sql</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">DELIMITER && drop procedure if exists SP_QueryData;Create Procedure SP_QueryData(IN stuid varchar(16))READS SQL DATA BEGIN SET @sql = NULL;SET @stuid = NULL;SELECT GROUP_CONCAT(DISTINCT CONCAT( 'MAX(IF(c.coursenm = ''', c.coursenm, ''', s.scores, 0)) AS ''', c.coursenm, '\'' ) ) INTO @sqlFROM courses c; SET @sql = CONCAT('Select st.stuid, st.stunm, ', @sql, ' From student st Left Join score s On st.stuid = s.stuid Left Join courses c On c.courseno = s.courseno'); IF stuid is not null and stuid <> '' thenSET @stuid = stuid;SET @sql = CONCAT(@sql, ' Where st.stuid = \'', @stuid, '\'');END IF; SET @sql = CONCAT(@sql, ' Group by st.stuid'); PREPARE stmt FROM @sql;EXECUTE stmt;DEALLOCATE PREPARE stmt; END && DELIMITER ;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>调用也很简单，直接用下面Sql就OK啦</p><pre class="line-numbers language-mysql" data-language="mysql"><code class="language-mysql">CALL `SP_QueryData`('1005');<span aria-hidden="true" class="line-numbers-rows"><span></span></span></code></pre><p>效果如图：</p><p><img src="https://cjx27.coding.net/p/resources/d/image/git/raw/master/LineToRow6.png" alt="效果图"></p></li></ul><p>好啦，以上就是我这次MySQL行转列的实现过程，希望大家看完都有所收益！！！</p>]]></content>
<categories>
<category> 技术 </category>
</categories>
<tags>
<tag> Mysql </tag>
<tag> SQL </tag>
</tags>
</entry>
<entry>
<title>Redis事件处理</title>
<link href="/article/2023/7367/"/>
<url>/article/2023/7367/</url>
<content type="html"><![CDATA[<h1 id="2-事件处理"><a href="#2-事件处理" class="headerlink" title="2 事件处理"></a>2 事件处理</h1><h2 id="2-1-I-O-多路复用"><a href="#2-1-I-O-多路复用" class="headerlink" title="2.1 I/O 多路复用"></a>2.1 I/O 多路复用</h2><p>Redis 处理速度快的原因之一就是通过 I/O 多路复用, 来处理大量的请求。<br>I/O 多路复用, 也叫做事件驱动模型。IO 多路复用是一种同步 IO 模型，实现一个线程可以监视多个文件句柄。一旦某个文件句柄就绪，就能够通知应用程序进行相应的读写操作,<br>没有文件句柄就绪时会阻塞应用程序，交出 CPU。多路是指网络连接，复用指的是同一个线程。</p><p>I/O 多路复用的实现方式有很多, Redis 支持 4 种模式: select, epoll, keueuq, evport。 在使用时, Redis 会根据系统的支持情况等, 从中选择一种。</p><p>这里以 epoll 为例。<br>epoll 是 Linux 内核为处理大量并发网络连接而提出的解决方案, 能显著提升系统 CPU 利用率。<br>epoll 的使用非常简单, 总共只有 3 个 API:</p><blockquote><ol><li>epoll_create 函数创建一个 epoll 专用的文件描述符, 用于后续 epoll 相关 API 调用</li><li>epoll_ctl 函数向 epoll 注册, 修改或删除需要监控的事件</li><li>epoll_wait 函数会阻塞进程, 直到监控的若干网络连接有事件发生</li></ol></blockquote><pre class="line-numbers language-C" data-language="C"><code class="language-C">int epoll_create(int size)<span aria-hidden="true" class="line-numbers-rows"><span></span></span></code></pre><p>入参是一个 int 类型的 size, 通知内核程序期望注册的网络连接数目，内核以此判断初始分配空间大小。 在 Linux 2.6.8 版本以后, 内核动态分配空间, 此参数会被忽略。<br>出参返回 epoll 专用的文件描述符。不再使用时应该及时关闭此文件描述符。</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event)<span aria-hidden="true" class="line-numbers-rows"><span></span></span></code></pre><p>入参有 4 个:</p><blockquote><ol><li>int 类型的 epfd, 函数 epoll_create 返回的 epoll 文件描述符</li><li>int 类型的 op, 需要进行的操作，EPOLL_CTL_ADD 表示注册事件， EPOLL_CTL_MOD 表示修改网络连接事件，EPOLL_CTL_DEL 表示删除事件, 等等其他事件</li><li>int 类型的 fd：网络连接的 socket 文件描述符</li><li>epoll_event 类型的 event：需要监控的事件</li></ol></blockquote><p>epoll_event 的定义</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">struct epoll_event { __uint32_t events; epoll_data_t data; };typedef union epoll_data { void *ptr; int fd; __uint32_t u32; __uint64_t u64; } epoll_data_t;<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>其中 events 表示需要监控的事件类型，比较常用的是 EPOLLIN 文件描述符可读事件，EPOLLOUT 文件描述符可写事件;<br>data 保存与文件描述符关联的数据。</p><p>epoll_ctl 函数执行成功时返回 0, 否则返回 -1, 错误码设置在全局变量 errno 中。</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">int epoll_wait(int epfd,struct epoll_event * events, int maxevents, int timeout)<span aria-hidden="true" class="line-numbers-rows"><span></span></span></code></pre><p>入参有 3 个:</p><blockquote><ol><li>int 类型的 epfd, 函数 epoll_create 返回的 epoll 文件描述符</li><li>epoll_event 类型的 epoll_event, 作为输出参数使用，用于回传已触发的事件数组</li><li>int 类型的 maxevents, 表示每次能处理的最大事件数目</li><li>int 类型的 timeout, epoll_wait 函数阻塞超时时间, 如果超过 timeout 时间还没有事件发生, 函数不再阻塞直接返回, 当 timeout 等于 0 时函数立即返回, timeout 等于 -1 时函数会一直阻塞直到有事件发生</li></ol></blockquote><h2 id="2-2-AE"><a href="#2-2-AE" class="headerlink" title="2.2 AE"></a>2.2 AE</h2><p>上面知道 Redis 底层可以使用 4 中 I/O 多路复用模型, 在启动时会根据系统的情况选择其中一种。<br>但是不同的 I/O 多路复用模型的 API 是不同的, 为了上层的调用方便, Redis 对 4 种方式进行了封装,<br>自制了一个异步事件库 “<strong>A simple event-driven programming library</strong>“, 简称 “ae”。</p><p>涉及的文件有</p><table><thead><tr><th align="center">文件名</th><th align="center">用途</th></tr></thead><tbody><tr><td align="center">ae_select.c</td><td align="center">select 的使用封装</td></tr><tr><td align="center">ae_kqueue.c</td><td align="center">kqueue 的使用封装</td></tr><tr><td align="center">ae_evport.c</td><td align="center">evport 的使用封装</td></tr><tr><td align="center">ae_epoll.c</td><td align="center">epoll 的使用封装</td></tr><tr><td align="center">ae.h</td><td align="center">AE 事件库接口定义</td></tr><tr><td align="center">ae.c</td><td align="center">AE 事件库实现</td></tr></tbody></table><p>4 个 I/O 多路复用模型文件只需要封装出下面几个函数</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">struct aeApiState;static int aeApiCreate(aeEventLoop *eventLoop);static int aeApiResize(aeEventLoop *eventLoop, int setsize);static void aeApiFree(aeEventLoop *eventLoop);static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask);static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask);static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp);static char *aeApiName(void);<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>后面的 AE 库就能通过调用这几个 API 达到封装的效果, 上层调用 AE 内对应的方法就能调用到 I/O 多路复用模型的方法, 而不需要关心底层相关的 API。</p><h3 id="2-2-1-AE-事件模型"><a href="#2-2-1-AE-事件模型" class="headerlink" title="2.2.1 AE 事件模型"></a>2.2.1 AE 事件模型</h3><p>Redis 服务器是典型的事件驱动程序，而事件又分为文件事件 (socket 的可读可写事件) 与时间事件 (定时任务) 两大类。<br>所以文件事件 ( IO 事件) 和时间事件就足以支撑服务端的全部功能。</p><h4 id="2-2-1-1-文件事件"><a href="#2-2-1-1-文件事件" class="headerlink" title="2.2.1.1 文件事件"></a>2.2.1.1 文件事件</h4><p>AE 中和文件事件相关的定义和接口: </p><pre class="line-numbers language-C" data-language="C"><code class="language-C">#define AE_NONE 0#define AE_READABLE 1#define AE_WRITABLE 2#define AE_FILE_EVENTS 1/** 读写时间时执行的函数 */typedef void aeFileProc(struct aeEventLoop *eventLoop, int fd, void *clientData, int mask);/** 文件事件定义 */typedef struct aeFileEvent { /** 文件类型, AE_READABLE (1, 表示读事件) 和 AE_WRITABLE (2, 表示写事件) 其中一个*/ int mask; /** 读事件时执行的函数 */ aeFileProc *rfileProc; /** 写事件时执行的函数 */ aeFileProc *wfileProc; /** 客户端数据 */ void *clientData;} aeFileEvent;/** 文件事件的创建函数 */int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask, aeFileProc *proc, void *clientData);/** 文件事件的删除函数 */void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask);/** 获取文件事件 */int aeGetFileEvents(aeEventLoop *eventLoop, int fd);<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><h4 id="2-2-1-2-时间事件"><a href="#2-2-1-2-时间事件" class="headerlink" title="2.2.1.2 时间事件"></a>2.2.1.2 时间事件</h4><p>AE 中和时间事件相关的定义和接口</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">#define AE_TIME_EVENTS 2/** 时间事件执行函数 */typedef int aeTimeProc(struct aeEventLoop *eventLoop, long long id, void *clientData);/** 时间事件从链表中删除时执行的函数 */typedef void aeEventFinalizerProc(struct aeEventLoop *eventLoop, void *clientData);/** 时间事件定义 */typedef struct aeTimeEvent { /** 事件 id, 不断递增的 */ long long id; /** 事件执行的时间 (秒数) */ long when_sec; /** 事件执行的时间 (毫秒数) */ long when_ms; /** 执行的函数 */ aeTimeProc *timeProc; /** 时间事件从链表中删除时执行的函数, 非必须 */ aeEventFinalizerProc *finalizerProc; /** 客户端数据 */ void *clientData; /** 上一个时间事件 */ struct aeTimeEvent *prev; /** 下一个时间事件 */ struct aeTimeEvent *next;} aeTimeEvent;/** 时间事件创建 */long long aeCreateTimeEvent(aeEventLoop *eventLoop, long long milliseconds, aeTimeProc *proc, void *clientData, aeEventFinalizerProc *finalizerProc);/** 时间文件的删除 */int aeDeleteTimeEvent(aeEventLoop *eventLoop, long long id);<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><h3 id="2-2-2-AE-事件轮询"><a href="#2-2-2-AE-事件轮询" class="headerlink" title="2.2.2 AE 事件轮询"></a>2.2.2 AE 事件轮询</h3><p>在上面的事件 API 中, 可以看到一个很重要的属性 aeEventLoop, 无论是事件的创建, 获取, 删除的操作都需要依赖到这个属性。<br>其实这个属性是对<strong>整个事件轮询</strong>的封装, 如整理时间循环的结束状态， 注册的事件等。 </p><p>aeEventLoop 相关的定义和接口</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">typedef struct aeEventLoop { /** 当前注册的最大文件描述符 */ int maxfd; /** 文件事件数组的容量 */ int setsize; /** 时间事件的下一个 id */ long long timeEventNextId; /** 用于检测系统时钟偏差 */ time_t lastTime; /** 注册的事件数组指针, 类型为数组 */ aeFileEvent *events; /** 存储被触发的文件事件, 在轮询中, 会调用对应的多路复用的 API, 找到所有的事件中满足条件的, 放到这里面*/ aeFiredEvent *fired; /** 时间事件指针， 类型为双向链表，新事件添加到头部 */ aeTimeEvent *timeEventHead; /** 识事件循环是否结束, 控制着整个时间轮询的结束 */ int stop; /** I/O 多路复用封装的 API 引用 */ void *apidata; /** Redis 服务器需要阻塞等待文件事件的发生，进程阻塞之前会调用 beforesleep 函数，进程因为某种原因被唤醒之后会调用 aftersleep 函数 */ aeBeforeSleepProc *beforesleep; aeBeforeSleepProc *aftersleep;} aeEventLoop;/** 创建 aeEventLoop */aeEventLoop *aeCreateEventLoop(int setsize);/** 删除 aeEventLoop */void aeDeleteEventLoop(aeEventLoop *eventLoop);/** 设置 aeEventLoop 的 2 个回调函数 */void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep);void aeSetAfterSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *aftersleep);/** 获取 aeEventLoop 的 setSize 值 */int aeGetSetSize(aeEventLoop *eventLoop);/** 重新设置 aeEventLoop 的 setSize 值*/int aeResizeSetSize(aeEventLoop *eventLoop, int setsize);/** 启动事件轮询 */void aeMain(aeEventLoop *eventLoop);/** 等待毫秒，直到给定的文件描述符变为 可写/可读/异常 */int aeWait(int fd, int mask, long long milliseconds);/** 执行事件轮询对象中对应的事件 */int aeProcessEvents(aeEventLoop *eventLoop, int flags);<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>通过上面几个的 API 大体可以知道整个 AE 库的执行过程</p><blockquote><ol><li>通过 aeCreateEventLoop 函数创建出一个事件轮询对象 aeEventLoop</li><li>通过 aeMain 启动整个数据轮询, 也就是启动一个死循环</li><li>在 aeEventLoop 的死循环内会不断的调用 aeProcessEvents 尝试去处理事件</li><li>调用 aeCreateFileEvent 和 aeCreateTimeEvent 创建事件, 放到 aeEventLoop 中</li></ol></blockquote><h4 id="2-2-2-1-AE-的执行过程"><a href="#2-2-2-1-AE-的执行过程" class="headerlink" title="2.2.2.1 AE 的执行过程"></a>2.2.2.1 AE 的执行过程</h4><p>aeEventLoop 的启动</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">void aeMain(aeEventLoop *eventLoop) { eventLoop->stop = 0; // 只要 aeEventLoop 的 stop 表示不为 0 while (!eventLoop->stop) { // 有配置 beforesleep 事件就执行 if (eventLoop->beforesleep != NULL) eventLoop->beforesleep(eventLoop); // 进行事件处理 // AE_ALL_EVENT = (AE_FILE_EVENTS|AE_TIME_EVENTS) = 1 | 2 = 01 | 10 = 11 = 3 // AE_CALL_AFTER_SLEEP = 8 // AE_ALL_EVENTS|AE_CALL_AFTER_SLEEP = 3 | 8 = 11 | 1000 = 1011 aeProcessEvents(eventLoop, AE_ALL_EVENTS|AE_CALL_AFTER_SLEEP); }}<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>事件处理</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">int aeProcessEvents(aeEventLoop *eventLoop, int flags) { int processed = 0, numevents; /** flags 中不包含时间事件和文件事件 */ if (!(flags & AE_TIME_EVENTS) && !(flags & AE_FILE_EVENTS)) return 0; /** 最大文件描述符不等于 - 1 (aeEventLoop 初始时 maxfd = -1), * 往 aeEventLoop 添加文件事件时, maxfd 会等于 原本的 maxfd 和入参的 fd 中的比较大的那个, maxfd != -1 表示有文件事件 * flags 中包含时间事件 && flags 中不包含 AE_DONT_WAIT */ if (eventLoop->maxfd != -1 || ((flags & AE_TIME_EVENTS) && !(flags & AE_DONT_WAIT))) { int j; aeTimeEvent *shortest = NULL; struct timeval tv, *tvp; // 省略 tvp 的计算 .... // 从中获取符合条件的事件数 numevents = aeApiPoll(eventLoop, tvp); // 设置了需要触发回调，同时配置了回调函数 if (eventLoop->aftersleep != NULL && flags & AE_CALL_AFTER_SLEEP) eventLoop->aftersleep(eventLoop); for (j = 0; j < numevents; j++) { // 获取触发的事件 aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd]; int mask = eventLoop->fired[j].mask; int fd = eventLoop->fired[j].fd; int rfired = 0; /** 执行读回调函数 */ if (fe->mask & mask & AE_READABLE) { rfired = 1; fe->rfileProc(eventLoop, fd, fe->clientData, mask); } /** 执行写回调函数 */ if (fe->mask & mask & AE_WRITABLE) { if (!rfired || fe->wfileProc != fe->rfileProc) fe->wfileProc(eventLoop, fd, fe->clientData, mask); } processed++; } } /** 执行定时器事件 */ if (flags & AE_TIME_EVENTS) processed += processTimeEvents(eventLoop); return processed;}// 时间事件处理static int processTimeEvents(aeEventLoop *eventLoop) { int processed = 0; aeTimeEvent *te, *prev; long long maxId; time_t now = time(NULL); // 当前的时间小于当前事件轮询的上次的时间, // 时间异常, 系统时间修改过，为防止定时器永远无法执行，将所有的定时器设置为立即执行 if (now < eventLoop->lastTime) { te = eventLoop->timeEventHead; while(te) { te->when_sec = 0; te = te->next; } } eventLoop->lastTime = now; prev = NULL; // 重试赋值为时间事件链表的头部 te = eventLoop->timeEventHead; maxId = eventLoop->timeEventNextId - 1; // 遍历整个时间时间链表 while(te) { long now_sec, now_ms; long long id; // 如果事件的 id = AE_DELETED_EVENT_ID (-1), 进行清除, 然后处理下一个, 删除失效定时器 // 逻辑省略 ...... // 事件的 id 大于当前的时间事件的最大 id, 跳过， if (te->id > maxId) { te = te->next; continue; } // 获取当前的时间单位: 秒和毫秒 aeGetTime(&now_sec, &now_ms); // 当前的时间 > 事件的执行时间 || 当前的时间 >= 事件的执行时间 if (now_sec > te->when_sec || (now_sec == te->when_sec && now_ms >= te->when_ms)) { int retval; id = te->id; // 调用事件身上的执行函数 // retval 为此时间事件下次应该被触发的时间，单位为毫秒，且是一个相对时间，即从当前时间算起，retval 毫秒后此时间事件会被触发 retval = te->timeProc(eventLoop, id, te->clientData); processed++; // 执行返回值 ！= -1 if (retval != AE_NOMORE) { // 更新 当前的时间事件的时间为 retval/1000 + 当前的时间 aeAddMillisecondsToNow(retval,&te->when_sec,&te->when_ms); } else { // 将 id 设置为 -1, 下次执行会被删除 te->id = AE_DELETED_EVENT_ID; } } te = te->next; } return processed;}<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><h2 id="2-3-大体的流程"><a href="#2-3-大体的流程" class="headerlink" title="2.3 大体的流程"></a>2.3 大体的流程</h2><blockquote><ol><li>先通过 aeCreateEventLoop 创建出一个 aeEventLoop</li><li>通过 aeCreateFileEvent 向 aeEventLoop 和 epoll 中注册 socket 事件</li><li>调用 aeMain 启动整个循环</li><li>循环中会不断的从 epoll 中获取符合条件的事件</li></ol></blockquote><h2 id="2-4-Redis-中的唯一一个时间事件"><a href="#2-4-Redis-中的唯一一个时间事件" class="headerlink" title="2.4 Redis 中的唯一一个时间事件"></a>2.4 Redis 中的唯一一个时间事件</h2><p>在 Redis 中只有一个时间时间, 这个时间时间是在 Redis 启动时进行创建的。<br>我们知道创建事件事件的函数为 <strong>aeCreateTimeEvent</strong>, 而这个函数的调用, 可以返照只有在 serve.c 的 <strong>initServer</strong> 中调用</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">void initServer(void) { // 执行时间为频率为 1 毫秒, 执行的函数为 serverCron, 客户端数据和结束执行函数都为空 if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) { // 创建失败, 就结束 serverPanic("Can't create event loop timers."); exit(1); }}<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>可以看出唯一的时间事件就这一个, 但是 Redis 总大量的定时器是怎么实现的呢, 其原因就在执行的函数 serverCron 中</p><pre class="line-numbers language-C" data-language="C"><code class="language-C">int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { // 1000/server.hz 可以看做是执行频率, xx 毫秒执行一次 // run_with_period() 是一个宏定义, 可以看做是一个函数 // run_with_period(ms) ==> if (ms<= 1000/server.hz || !(server.cronloops%(ms/(1000.server.hz)))), 判断为真, 就会执行里面的方法 // 配置的时间小于执行频率, 符合第一个条件, 直接执行 // 配置的时间大于执行频率, 那么可以求出配置的时间是多少个执行频率, 整个函数的执行次数 % 需要的个数 == 0 就可以执行了 run_with_period(100) { // 100 毫秒周期执行 } run_with_period(5000) { // 5000 毫秒周期执行 } ... // 清除超时客户端连接 lientsCron(); // 处理数据库 databasesCron(); // cronloops 当前函数执行的次数 server.cronloops++; // server.hz 表示 serverCron 函数的执行频率, 默认为 10, 最小为 1, 最大值为 500, 用户可以配置 // 1000/server.hz 的返回值会更新当前时间事件的触发时间, 比如 server.hz 的默认值 10, 1000/10 = 100 毫秒 // 第一次执行完 serverCron, 返回值为 100, 那么下次执行 serverCron 为 100 毫秒后 return 1000/server.hz;}<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><p>就一个定时器事件在执行所有的定时任务, 可以看出里面除了周期性的任务, 也包含着每次必定执行的任务。<br>上面所有的任务都可能存在一个任务执行耗时很长, 从而影响到整个定时器, 所有 Redis 中对这些任务都做了一些特殊处理, 保证整个定时器的的服务质量。<br>比如清除过期键的操作, 会设置一个超时时间, 给每次循环设置一个次数, 每次循环这么多次，就检查一下是否已经超时了, 没有继续循环, 超了就结束。</p><h2 id="2-5-参考"><a href="#2-5-参考" class="headerlink" title="2.5 参考"></a>2.5 参考</h2><p><a href="https://www.jianshu.com/p/da092472080e">Redis AE异步事件库实例分析</a></p>]]></content>
<categories>
<category> 技术 </category>
</categories>
<tags>
<tag> Redis </tag>
</tags>
</entry>
<entry>
<title>Redis实现分分布式锁</title>
<link href="/article/2023/13529/"/>
<url>/article/2023/13529/</url>
<content type="html"><![CDATA[<h1 id="Redis实现分布式锁"><a href="#Redis实现分布式锁" class="headerlink" title="Redis实现分布式锁"></a>Redis实现分布式锁</h1><h2 id="分布式锁一般有三种实现方式："><a href="#分布式锁一般有三种实现方式：" class="headerlink" title="分布式锁一般有三种实现方式："></a>分布式锁一般有三种实现方式：</h2><ol><li>基于数据库乐观锁实现；</li><li>基于Redis的分布式锁；</li><li>基于ZooKeeper的分布式锁。</li></ol><h2 id="基于Redis分布式锁实现实例"><a href="#基于Redis分布式锁实现实例" class="headerlink" title="基于Redis分布式锁实现实例"></a>基于Redis分布式锁实现实例</h2><p>♥废话不多说，直接上代码♥</p><hr><h3 id="代码实现"><a href="#代码实现" class="headerlink" title="代码实现"></a>代码实现</h3><ul><li><p>首先我们要通过Maven引入<code>Redis</code>开源组件，在<code>pom.xml</code>文件加入下面的代码：</p><pre class="line-numbers language-java" data-language="java"><code class="language-java"><span class="token generics"><span class="token punctuation"><</span>dependency<span class="token punctuation">></span></span> <span class="token generics"><span class="token punctuation"><</span>groupId<span class="token punctuation">></span></span>org<span class="token punctuation">.</span>springframework<span class="token punctuation">.</span>data<span class="token operator"><</span><span class="token operator">/</span>groupId<span class="token operator">></span> <span class="token generics"><span class="token punctuation"><</span>artifactId<span class="token punctuation">></span></span>spring<span class="token operator">-</span>data<span class="token operator">-</span>redis<span class="token operator"><</span><span class="token operator">/</span>artifactId<span class="token operator">></span> <span class="token generics"><span class="token punctuation"><</span>version<span class="token punctuation">></span></span><span class="token number">2.3</span><span class="token number">.3</span><span class="token punctuation">.</span><span class="token constant">RELEASE</span><span class="token operator"><</span><span class="token operator">/</span>version<span class="token operator">></span><span class="token operator"><</span><span class="token operator">/</span>dependency<span class="token operator">></span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span></span></code></pre></li><li><p>加锁设值</p><pre class="line-numbers language-java" data-language="java"><code class="language-java"><span class="token comment">/** * * @param key Redis的key * @param numOfMin key值失效时间 * @param supplier 函数式接口 存储的value * @return */</span><span class="token keyword">public</span> <span class="token class-name">String</span> <span class="token function">getTempValue</span><span class="token punctuation">(</span><span class="token class-name">String</span> key<span class="token punctuation">,</span> <span class="token keyword">int</span> numOfMin<span class="token punctuation">,</span> <span class="token class-name">Supplier</span><span class="token generics"><span class="token punctuation"><</span><span class="token class-name">String</span><span class="token punctuation">></span></span> supplier<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token class-name">String</span> value <span class="token operator">=</span> <span class="token function">getTempValue</span><span class="token punctuation">(</span>key<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token class-name">StringUtils</span><span class="token punctuation">.</span><span class="token function">isBlank</span><span class="token punctuation">(</span>value<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">// 削峰</span><span class="token class-name">String</span> mux <span class="token operator">=</span> <span class="token string">"getTempValue"</span> <span class="token operator">+</span> key<span class="token punctuation">;</span> <span class="token comment">// 判断是否获锁成功</span><span class="token keyword">boolean</span> store <span class="token operator">=</span> redisTemplate<span class="token punctuation">.</span><span class="token function">boundValueOps</span><span class="token punctuation">(</span>mux<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">setIfAbsent</span><span class="token punctuation">(</span><span class="token string">"1"</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">if</span><span class="token punctuation">(</span>store<span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token comment">// 设置这个锁的失效时间</span>redisTemplate<span class="token punctuation">.</span><span class="token function">expire</span><span class="token punctuation">(</span>mux<span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">,</span><span class="token class-name">TimeUnit</span><span class="token punctuation">.</span><span class="token constant">SECONDS</span><span class="token punctuation">)</span><span class="token punctuation">;</span>value <span class="token operator">=</span> supplier<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token function">setSimilarData</span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> value<span class="token punctuation">,</span> numOfMin<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token comment">// 操作完删除这个锁</span>redisTemplate<span class="token punctuation">.</span><span class="token function">delete</span><span class="token punctuation">(</span>mux<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span class="token keyword">else</span><span class="token punctuation">{</span><span class="token keyword">try</span> <span class="token punctuation">{</span><span class="token class-name">Thread</span><span class="token punctuation">.</span><span class="token function">sleep</span><span class="token punctuation">(</span><span class="token number">50</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">InterruptedException</span> e<span class="token punctuation">)</span> <span class="token punctuation">{</span>e<span class="token punctuation">.</span><span class="token function">printStackTrace</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span> <span class="token comment">// 递归调用</span>value <span class="token operator">=</span> <span class="token function">getTempValue</span><span class="token punctuation">(</span>key<span class="token punctuation">,</span> numOfMin<span class="token punctuation">,</span> supplier<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span class="token punctuation">}</span><span class="token keyword">return</span> value<span class="token punctuation">;</span><span class="token punctuation">}</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></span></code></pre><pre class="line-numbers language-java" data-language="java"><code class="language-java"><span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">setSimilarData</span><span class="token punctuation">(</span><span class="token class-name">String</span> key<span class="token punctuation">,</span> <span class="token class-name">String</span> value<span class="token punctuation">,</span> <span class="token keyword">int</span> time<span class="token punctuation">)</span> <span class="token punctuation">{</span><span class="token comment">// 存储到redis并设置过期时间</span>redisTemplate<span class="token punctuation">.</span><span class="token function">boundValueOps</span><span class="token punctuation">(</span>key<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span>value<span class="token punctuation">,</span> time<span class="token punctuation">,</span> <span class="token class-name">TimeUnit</span><span class="token punctuation">.</span><span class="token constant">MINUTES</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">}</span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span><span></span></span></code></pre></li></ul><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>如果你的项目中Redis是多机部署的，那么可以尝试使用<code>Redisson</code>实现分布式锁，这是Redis官方提供的Java组件。</p>]]></content>
<categories>
<category> 技术 </category>
</categories>
<tags>
<tag> Redis </tag>
</tags>
</entry>
<entry>
<title>Hexo 结合coding部署详解</title>
<link href="/article/2022/40152/"/>
<url>/article/2022/40152/</url>
<content type="html"><![CDATA[<h1 id="Hexo-结合coding部署详解"><a href="#Hexo-结合coding部署详解" class="headerlink" title="Hexo 结合coding部署详解"></a>Hexo 结合coding部署详解</h1><h2 id="提交文件放置目录"><a href="#提交文件放置目录" class="headerlink" title="提交文件放置目录"></a>提交文件放置目录</h2><p>对应静态仓库下</p><h2 id="部署命令"><a href="#部署命令" class="headerlink" title="部署命令"></a>部署命令</h2><pre class="line-numbers language-bash" data-language="bash"><code class="language-bash">hexo cleanhexo generate 或者 hexo ghexo deploy 或者 hexo<span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span></span></code></pre><h2 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h2><p><a href="%22https://www.jianshu.com/p/85192626faf3%22">具体实现</a></p>]]></content>
<categories>
<category> 技术 </category>
</categories>
<tags>
<tag> Hexo </tag>
<tag> JavaScript </tag>
</tags>
</entry>
</search>