Memory management in Python

Python 的内存管理模型,在 Python 中,一切都是 object (对象)。为了更加高效的管理内存,设计了三种不同的数据结构,Block, Pool, Arena. Block 为最小的数据结构,从 8 到 512 bytes,但必须是 8 的整数,被划分为 64 种不同的大小。Pool 则是大小相同的 Block 的集合,一般情况下 Pool 的大小等于 Memory page ,4 kb。将 Pool 限定在只包含固定大小的 Block 是为了方便 fragmentation (碎片)管理,当一个 block 被消除后,随即可以用一个新的 block 填充上来。Pool 是由 doubly linked list (双向链表)链接而成。Arena 是堆上分配的 256k 大小的内存块,为 64 个 pool 提供存储空间。Python 的 small object manager 很少将内存归还给操作系统,只有当Arena 中所有的 Pool 为空时,Arena 才会被释放,也只有短时间内有大量的临时 objects 时才会出现这种情况。所以,对于一些长期运行的 Python 进程,可能会因此占用大量未被释放的内存。

Memory management in Python

comments powered by Disqus

Tags

abcs accept acid activemq affinity algorithm allocation android array async aws b+tree b-tree backoff benchmark best-practices bfs big-o bigquery bind bitcount blog break broker bubble buffer cache cap cert cgroups channel citus class classmethod cluster concurrency config consumer container cookie cors crawler cronjob csrf ctr data-science data-structure database datadog dataflow datascience decorator defer dfs distributed django dns docker double-shipping drf ecosia elastic-search enumerate epoll extra facebook-pixel financial-report flask flutter forward-proxy freelance frp garbage-collector gc gcp generator get gil git golang goroutine graphql ha handbook haproxy hash hash-slot hashring hashtable hpa http http-auth http-proxy http_proxy https index innodb instagram intergration interview ios javascript jinja2 jobboard jwt k8s kafka kibana kqueue label lambda layer4 layer7 lean levels.io linked-list linux list listen loadbalancer logs long-tail lru marketing master matplotlib memory merge metaclass metaprogramming metrics metrics-server microservices mitm model mq myisam mysql namespace nat netflix network-extension nginx nodejs nomad nosql npm oodesign openssl optimization orm pandas parallelism parkinglot patroni permission pg pipeline pixelme post postgresql postresql prefetch_related prerender process proxy proxycommand put pvm python queue rabbitmq rbac react-native rebase redis redis-cluster replication resource rest restfulapi retargeting retry revenue reverse-proxy rocketmq saas scaleable search-engine security select seo serverless service session set shadosocks shadowsocks shard sharding shell shopify sigint signal sigterm slack slave slow-query sniper sns socket socks5 source-code spa sql sqlalchemy sqs ssh ssl ssl-pinning stack startup state stateful stateless staticmethod string struct swift switch syscall system-design systemctl tcp tcp-proxy thread tmpreaper token traefik trustkit tunning type typeform udp variable vc vpn vuejs web web-development where yarn zset 削峰 宽索引 异步 窄索引 解耦 跨域 跳板机



Archives

2019 (137)
0001 (1)