Q&F · Django · 看云

### Q&F源码这两个查询数据结构基本上在Django里面无处不在，今天就看看这个里面的代码首先Q是继承自django.utils.tree里面的Node类的，这里先分析下Node的源码 #### Q的基本逻辑操作 ~~~ class Node(object): """ A single internal node in the tree graph. A Node should be viewed as a connection (the root) with the children being either leaf nodes or other Node instances. """ # Standard connector type. Clients usually won't use this at all and # subclasses will usually override the value. default = 'DEFAULT' def __init__(self, children=None, connector=None, negated=False): """ Constructs a new Node. If no connector is given, the default will be used. """ self.children = children[:] if children else [] self.connector = connector or self.default self.negated = negated ~~~ 这一段介绍相当清楚了，default类属性基本用不到，且会被其他子类例如Q覆盖，children这一行的赋值就是简单的数组复制罢了，另外两个变量也没什么难以理解的 ~~~py # We need this because of django.db.models.query_utils.Q. Q. __init__() is # problematic, but it is a natural Node subclass in all other respects. @classmethod def _new_instance(cls, children=None, connector=None, negated=False): """ This is called to create a new instance of this class when we need new Nodes (or subclasses) in the internal code in this class. Normally, it just shadows __init__(). However, subclasses with an __init__ signature that is not an extension of Node.__init__ might need to implement this method to allow a Node to create a new instance of them (if they have any extra setting up to do). """ obj = Node(children, connector, negated) obj.__class__ = cls return obj ~~~ 这个方法是用来给重新定义了__init__函数并且没有调用Node的__init__的子类来使用的，因为当内部方法调用时有很多情况需要Node实例接下来几个都是常被定义的魔术方法，没什么可讲的，值得提下的就是len和contains函数 ~~~ def __len__(self): """ The size of a node if the number of children it has. """ return len(self.children) def __contains__(self, other): """ Returns True is 'other' is a direct child of this instance. """ return other in self.children ~~~ 这两个函数表明了一个Node的大小是由它的children属性决定的，同时在检测包含性关系时也是看另外一个Node是否在它的chidren数组中 了解到这里之后暂时打住，我们再开始看Q的代码 ~~~py class Q(tree.Node): """ Encapsulates filters as objects that can then be combined logically (using `&` and `|`). """ # Connection types AND = 'AND' OR = 'OR' default = AND def __init__(self, *args, **kwargs): super(Q, self).__init__(children=list(args) + list(kwargs.items())) def _combine(self, other, conn): if not isinstance(other, Q): raise TypeError(other) obj = type(self)() obj.connector = conn obj.add(self, conn) obj.add(other, conn) return obj def __or__(self, other): return self._combine(other, self.OR) def __and__(self, other): return self._combine(other, self.AND) ~~~ AND, OR, default三个类属性明显就是覆盖父类里的default属性用的，__init__函数也没有特殊之处，就是把传入的参数全部粗暴的数组化赋给children属性，这里需要注意的是，我们一般使用Q的时候都是形如下面这样的调用 ~~~ Q(param=value) ~~~ 而很少会出现直接扔一个位置参数进去的情况，以我个人目前的经验来看，从来没看到过实际代码里面给Q传入过位置参数，所以这里暂时不考虑有args的情况 再就是_combine函数了，名字很容易理解，两个参数也是很容易看出来用途 ~~~ if not isinstance(other, Q): raise TypeError(other) ~~~ 这里说明了Q只能和Q合并，其他Node的子类也不行然后就是利用元类创建一个新的空Q对象，同时将传入的连接类型赋予obj，下面两行的操作就是利用这个空Q对象对self和other两个Q进行合并同时返回obj，具体的连接方法就是Node类的add函数了，这里再回到Node类分析下add函数 ~~~py def add(self, data, conn_type, squash=True): """ Combines this tree and the data represented by data using the connector conn_type. The combine is done by squashing the node other away if possible. This tree (self) will never be pushed to a child node of the combined tree, nor will the connector or negated properties change. The function returns a node which can be used in place of data regardless if the node other got squashed or not. If `squash` is False the data is prepared and added as a child to this tree without further logic. """ if data in self.children: return data if not squash: self.children.append(data) return data if self.connector == conn_type: # We can reuse self.children to append or squash the node other. if (isinstance(data, Node) and not data.negated and (data.connector == conn_type or len(data) == 1)): # We can squash the other node's children directly into this # node. We are just doing (AB)(CD) == (ABCD) here, with the # addition that if the length of the other node is 1 the # connector doesn't matter. However, for the len(self) == 1 # case we don't want to do the squashing, as it would alter # self.connector. self.children.extend(data.children) return self else: # We could use perhaps additional logic here to see if some # children could be used for pushdown here. self.children.append(data) return data else: obj = self._new_instance(self.children, self.connector, self.negated) self.connector = conn_type self.children = [obj, data] return data ~~~ 以Q的__or__为例来进行分析，这里传入的self就是data形参，conn就是OR连接类型，第一个判断 ~~~ if data in self.children: return data ~~~ 因为调用add函数的obj是个空Q，所以除非本身self这个Q也是个空Q，不然这里肯定不存在包含关系 ~~~py if not squash: self.children.append(data) return data ~~~ 这里也不可能命中逻辑，因为都没有传入过第三个参数squash ~~~py if self.connector == conn_type: ~~~ 因为在_combine函数中已经设置过obj的connector属性了，且就是使用conn参数来设置的，所以在_combine中对add的调用必然命中此逻辑，接下来 ~~~py if (isinstance(data, Node) and not data.negated and (data.connector == conn_type or len(data) == 1)): self.children.extend(data.children) return self ~~~ 第一个判断一定是真，但是后面两个判断不一定了，当传入的是~Q这样的格式的时候就不会走这里的逻辑了，这里能接受的逻辑就是self这个Q的连接符和要做的操作连接符是一样的情况，也就是AND和AND，OR和OR，如果不是这种情况那就只能接受一个长度的和它本身连接符不一样的Q对象，所谓的一个长度，也就是里面只有一个对一个参数的筛选条件的情况，这种情况是把要合并的Q的children属性当做额外的数组并进当前的数组 ~~~py else: # We could use perhaps additional logic here to see if some # children could be used for pushdown here. self.children.append(data) return data ~~~ 而这里就很明显了，当前面的两个条件有一个不满足时，即当连接的Q是~Q或者两个连接符不一致且这个要连接的Q的children还大于1时，就执行上面这种逻辑，即把要连接的Q当做一个元素加进当前的children数组，这里总结一下两种情况，第一种情况 ~~~ a = Q(param1=1) b = Q(param2=2, param3=3) c = Q(param4=4) >>> a & b >>> a | c ~~~ 这里的a & b和a | c都满足第一个判断 ~~~ a = Q(param1=1) b = Q(param2=2, param3=3) c = ~Q(param4=4) >>> a | b >>> a & c ~~~ 这里的两种情况都满足第二种判断，这里再分析一下这四种情况的children和connector分别为多少 **第一种情况** ~~~ obj.children = [('param1', 1), ('param2', 2), ('param3', 3)] obj.connector = AND ~~~ **第二种情况** ~~~ obj.children = [('param1', 1), ('param4', 4)] obj.connector = OR ~~~ **第三种情况** ~~~ obj.children = [a, b] obj.connector = OR ~~~ **第四种情况** ~~~ obj.children = [a, c] obj.connector = AND 此时第二个元素的Q的negated属性为True ~~~ 这里还有一种最外层的else情况即self.connector != conn_type这种情况，然而这种情况至少在Q的与，或操作逻辑中是不存在的，因为与或操作的共同逻辑_combine方法都是通过创建一个空壳Q对象并将当前逻辑操作的操作符直接赋予这个空壳Q对象来进行运算的 不过非逻辑就不是如此了 ~~~py def __invert__(self): obj = type(self)() obj.add(self, self.AND) obj.negate() return obj ~~~ 非逻辑没有对空壳Q对象的连接符予以操作，所以这里obj的connector属性应该是‘DEFAULT'，自然就进入了Node类的else逻辑 ~~~py else: obj = self._new_instance(self.children, self.connector, self.negated) self.connector = conn_type self.children = [obj, data] return data ~~~ 这时候又创建了一个新的Node对象obj，再来看Node的_new_instance方法 ~~~py @classmethod def _new_instance(cls, children=None, connector=None, negated=False): """ This is called to create a new instance of this class when we need new Nodes (or subclasses) in the internal code in this class. Normally, it just shadows __init__(). However, subclasses with an __init__ signature that is not an extension of Node.__init__ might need to implement this method to allow a Node to create a new instance of them (if they have any extra setting up to do). """ obj = Node(children, connector, negated) obj.__class__ = cls return obj ~~~ 还是以实际调用的视角来分析一下这个类方法，children参数因为是空壳Q对象的属性，所以children是一个空数组，connector参数为DEFAULT值，negated此时还是False，所以这里其实还是创建了一个空壳对象，只不过这个空壳对象通过__class__属性强行从Node类变成了Q类，然后__invert__方法中的Q又将自己的连接符从DEFAULT变成and，同时这个空壳Q将add函数中创建的空壳Q对象obj和最开始执行这一套逻辑的原始Q对象一起纳入了自己的children数组，最后再讲negated属性反过来完成整套逻辑 总结：这里可以看出Q的代码虽然不长，但是逻辑其实很复杂，因为出现了多次子对象倒挂父对象同时进入深层调用的复杂情况，总结起来可以归纳为最表层即与或非三种逻辑中创建的空壳Q对象obj最后才是真正返回的对象，也就是说，**Q对象每进行一次逻辑操作都会返回新的Q对象**，这点上和QuerySet的特性是保持一致的 #### Q的其他几个函数(较少被用到) 接下来是Q剩下的两个函数，上面的几个魔术方法都是基本的逻辑合并，但是Q究竟是如何在底层配合QuerySet共同实现Django的ORM功能的呢？那很明显就是下面这两个方法的功劳了 ~~~py def resolve_expression(self, query=None, allow_joins=True, reuse=None, summarize=False, for_save=False): # We must promote any new joins to left outer joins so that when Q is # used as an expression, rows aren't filtered due to joins. clause, joins = query._add_q(self, reuse, allow_joins=allow_joins, split_subq=False) query.promote_joins(joins) return clause ~~~ 此函数的主体可以说基本没有内容，是一个完全依赖Query实例对象的函数