nginx-lua-module

2019-01-01  本文已影响0人  m风满楼

Description 简述

Almost all the Nginx modules can be used with this ngx_lua module by means of ngx.location.capture or ngx.location.capture_multi but it is recommended to use those lua-resty-* libraries instead of creating subrequests to access the Nginx upstream modules because the former is usually much more flexible and memory-efficient.
几乎所有的nginx模块都能通过ngx.location.capture或ngx.location.catpure_multi使用但是推荐使用lua-resty-*库而不是创建一个子请求去访问nginx上游服务,因为这种形式更加灵活和内存高效。

The Lua interpreter or LuaJIT instance is shared across all the requests in a single nginx worker process but request contexts are segregated using lightweight Lua coroutines
在单个nginx work进程中,所有的request共享Lua或者LuaJIT实例。但是请求上下文通过轻量级的Lua协程隔离

Loaded Lua modules persist in the nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads.
载入的lua模块在nginx进程级别持续存在,即使在负载较重的情况下,lua也占用很少内存。

This module is plugged into NGINX's "http" subsystem so it can only speaks downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, and etc). If you want to do generic TCP communications with the downstream clients, then you should use the ngx_stream_lua module instead which has a compatible Lua API
这些模块在nginx http子系统中以插件形式提供。所以你可以使用HTTP协议族调用下游服务。如果你想使用通用TCP协议和下游沟通,需要使用ngx-stream-lua模块中提供的兼容Lua API。

Typical Uses 典型用途

Just to name a few:

Mashup'ing and processing outputs of various nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua,
在Lua中混合和处理各种nginx上游输出

doing arbitrarily complex access control and security checks in Lua before requests actually reach the upstream backends
在请求实际到达上游后端之前,在Lua中进行任意复杂的访问控制和安全检查

manipulating response headers in an arbitrary way (by Lua)
以任意方式操纵响应头

fetching backend information from external storage backends (like redis, memcached, mysql, postgresql) and use that information to choose which upstream backend to access on-the-fly,
从外部后端存储获取信息并使用该信息选择哪个后端上游服务

coding up arbitrarily complex web applications in a content handler using synchronous but still non-blocking access to the database backends and other storage,
同步非阻塞的访问数据库后端或其他存储,在内容处理中编码实现任意复杂的应用。

doing very complex URL dispatch in Lua at rewrite phase,
using Lua to implement advanced caching mechanism for Nginx's subrequests and arbitrary locations.
在Lua rewrite阶段实现非常复杂的URL 分发,为Nginx子请求使用Lua实现高级缓存

The possibilities are unlimited as the module allows bringing together various elements within Nginx as well as exposing the power of the Lua language to the user. The module provides the full flexibility of scripting while offering performance levels comparable with native C language programs both in terms of CPU time as well as memory footprint. This is particularly the case when LuaJIT 2.x is enabled.
可能性是无限的,因为该模块允许将nginx的各种模块集合在一起并给将Lua语言的强大暴露给用户。

Other scripting language implementations typically struggle to match this performance level
其它脚本语言通常难以匹配这种性能级别。

The Lua state (Lua VM instance) is shared across all the requests handled by a single nginx worker process to minimize memory use.
Lua 虚拟机实例在每个工作进程的所有请求之间共享以减少内存占用。

INSTALLATION 安装。。。

Lua/LuaJIT bytecode support Lua字节码支持

As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as content_by_lua_file) support loading Lua 5.1 and LuaJIT 2.0/2.1 raw bytecode files directly.
v0.5.0rc32 版本开始, 所有 *_by_lua_file配置支持直接载入Lua或者LuaJIT字节码。

Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if you are using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown
请注意,LuaJIT 2.0 / 2.1使用的字节码格式与标准Lua 5.1解释器使用的格式不兼容。因此,如果您使用LuaJIT 2.0 / 2.1和ngx_lua,必须生成LuaJIT兼容的字节码文件。示例如下:

/path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc

Statically Linking Pure Lua Modules 静态链接纯Lua模块

When LuaJIT 2.x is used, it is possible to statically link the bytecode of pure Lua modules into the Nginx executable.
当使用LuaJIT2.x时,可以在nginx执行文件中直接静态连接纯Lua字节码模块。

Basically you use the luajit executable to compile .lua Lua module files to .o object files containing the exported bytecode data, and then link the .o files directly in your Nginx build.
基本上,您使用luajit可执行文件将.lua Lua模块文件编译为包含导出的字节码数据的.o对象文件,然后直接在Nginx构建中链接.o文件。

Below is a trivial example to demonstrate this. Consider that we have the following .lua file named foo.lua:
下面是一个简单的例子来证明这一点。考虑到我们有以下名为foo.lua的.lua文件

-- foo.lua
 local _M = {}

 function _M.go()
     print("Hello from foo")
 end

 return _M

And then we compile this .lua file to foo.o file
当我们编译该.lua文件为foo.o文件。
/path/to/luajit/bin/luajit -bg foo.lua foo.o

What matters here is the name of the .lua file, which determines how you use this module later on the Lua land. The file name foo.o does not matter at all except the .o file extension (which tells luajit what output format is used). If you want to strip the Lua debug information from the resulting bytecode, you can just specify the -b option above instead of -bg.
这里重要的是.lua文件的名称,它决定了你以后如何在Lua上使用这个模块。除了.o文件扩展名(告诉luajit使用了什么输出格式)之外,文件名foo.o完全没有关系。如果要从生成的字节码中去除Lua调试信
息,可以指定上面的-b选项而不是-bg。

Then when building Nginx or OpenResty, pass the --with-ld-opt="foo.o" option to the ./configure script:
当编译nginx的时候,通过在configure中添加 --with-ld-opt=''foo.o''选项来编译:
./configure --with-ld-opt="/path/to/foo.o" ...

Finally, you can just do the following in any Lua code run by ngx_lua:
最终,你可以在任意Lua代码中以下面的形式使用:

 local foo = require "foo"
 foo.go()

And this piece of code no longer depends on the external foo.lua file any more because it has already been compiled into the nginx executable.
这段代码不再依赖外部foo.lua文件,因为他已经被编译仅nginx可执行文件中了。

If you want to use dot in the Lua module name when calling require, as in
如果要在调用require时在Lua模块名称中使用dot,请参阅
local foo = reuire "resty.foo"
then you need to rename the foo.lua file to resty_foo.lua before compiling it down to a .o file with the luajit command-line utility.
然后你需要将foo.lua文件重命名为resty_foo.lua,然后使用luajit命令行实用程序将其编译为.o文件

It is important to use exactly the same version of LuaJIT when compiling .lua files to .o files as building nginx + ngx_lua. This is because the LuaJIT bytecode format may be incompatible between different LuaJIT versions. When the bytecode format is incompatible, you will see a Lua runtime error saying that the Lua module is not found.
在将.lua文件编译为.o文件时,使用完全相同版本的LuaJIT非常重要,因为它构建了nginx + ngx_lua。这是因为LuaJIT字节码格式在不同的LuaJIT版本之间可能不兼容。当字节码格式不兼容时,您将看到Lua运行时错误,指出未找到Lua模块。

When you have multiple .lua files to compile and link, then just specify their .o files at the same time in the value of the --with-ld-opt option. For instance,
如果要编译和链接多个.lua文件,则只需在--with-ld-opt选项的值中同时指定其.o文件。例如,
./configure --with-ld-opt="/path/to/foo.o /path/to/bar.o" ...

If you have just too many .o files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your .o files, as in
如果你有太多.o文件,那么在一个命令中将它们全部命名可能是不可行的。在这种情况下,您可以为.o文件构建静态库(或存档),如
ar rcus libmyluafiles.a *.o
then you can link the myluafiles archive as a whole to your nginx executable:
./configure \ --with-ld-opt="-L/path/to/lib -Wl,--whole-archive -lmyluafiles -Wl,--no-whole-archive"
然后你可以将myluafiles存档作为一个整体链接到你的nginx可执行文件。

where /path/to/lib is the path of the directory containing the libmyluafiles.a file. It should be noted that the linker option --whole-archive is required here because otherwise our archive will be skipped because no symbols in our archive are mentioned in the main parts of the nginx executable.
其中/ path / to / lib是包含libmyluafiles.a文件的目录的路径。应该注意的是,这里需要链接器选项--whole-archive,否则我们的存档将被跳过,因为在nginx可执行文件的主要部分中没有提到我们的存档中的符号。

Data Sharing within an Nginx Worker nginx进程间的数据共享

To globally share data among all the requests handled by the same nginx worker process, encapsulate the shared data into a Lua module, use the Lua require builtin to import the module, and then manipulate the shared data in Lua. This works because required Lua modules are loaded only once and all coroutines will share the same copy of the module (both its code and data). Note however that Lua global variables (note, not module-level variables) WILL NOT persist between requests because of the one-coroutine-per-request isolation design.
要在同一个nginx工作进程处理的所有请求之间全局共享数据,将共享数据封装到Lua模块中,使用Lua require builtin导入模块,然后在Lua中操作共享数据。这是有效的,因为所需的Lua模块只加载一次,并且所有协同程序将共享模块的相同副本(包括其代码和数据)。但请注意,由于每个请求的单协程隔离设计,Lua全局变量(注意,而不是模块级变量)不会在请求之间保持不变。

Here is a complete small example:
这里是一个完整的小例子。

 -- mydata.lua
 local _M = {}

 local data = {
     dog = 3,
     cat = 4,
     pig = 5,
 }

 function _M.get_age(name)
     return data[name]
 end

 return _M

The mydata module in this example will only be loaded and run on the first request to the location /lua, and all subsequent requests to the same nginx worker process will use the reloaded instance of the module as well as the same copy of the data in it, until a HUP signal is sent to the Nginx master process to force a reload. This data sharing technique is essential for high performance Lua applications based on this module.
此示例中的mydata模块仅在第一次请求时加载并运行到位置/ lua,对同一nginx工作进程的所有后续请求将重新加载使用模块的实例以及其中的数据,直到'HUP`信号被发送到Nginx主进程以强制重新加载。这种数据共享技术对于基于该模块的高性能Lua应用程序至关重要

Note that this data sharing is on a per-worker basis and not on a per-server basis. That is, when there are multiple nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers.
请注意,此数据共享基于每个worker而不是基于每个服务器。也就是说,当Nginx主机下有多个nginx工作进程时,数据共享不能越过这些工作进程之间的进程边界。

It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each nginx worker process as long as there is no nonblocking I/O operations in the middle of your calculations. As long as you do not give the control back to the nginx event loop and ngx_lua's light thread scheduler (even implicitly), there can never be any race conditions in between. For this reason, always be very careful when you want to share changeable data on the worker level. Buggy optimizations can easily lead to hard-to-debug race conditions under load.
通常建议以这种方式共享只读数据。只要在计算过程中存在nonblocking I / O操作,您还可以在每个nginx工作进程的所有并发请求之间共享可更改数据。只要你不将控制权交还给nginx事件循环和ngx_lua的轻线程调度程序(甚至隐式),就不会有任何竞争条件。因此,当您想要在worker级别共享可更改数据时,请务必非常小心。bug优化很容易导致在负载下难以调试的竞争条件

If server-wide data sharing is required, then use one or more of the following approaches:
如果需要服务器范围的数据共享,请使用以下一种或多种方法

  1. Use the [ngx.shared.DICT API provided by this module.
    使用ngx.shared.DICT API
  2. Use only a single nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine).
    使用单进程nginx和一个单服务器
  3. Use data storage mechanisms such as memcached, redis, MySQL or PostgreSQLassociated with this module comes with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms.
    使用 数据存储服务。

Known Issues 已知问题

TCP socket connect operation issues
Lua Coroutine Yielding/Resuming
Lua Variable Scope Lua变量作用域

Care must be taken when importing modules and this form should be used:
当导入模块时务必使用下面的形式

 local xxx = require('xxx')

而不是老的形式:

 require('xxx')

Here is the reason: by design, the global environment has exactly the same lifetime as the Nginx request handler associated with it. Each request handler has its own set of Lua global variables and that is the idea of request isolation. The Lua module is actually loaded by the first Nginx request handler and is cached by the require() built-in in the package.loaded table for later reference, and the module() builtin used by some Lua modules has the side effect of setting a global variable to the loaded module table. But this global variable will be cleared at the end of the request handler, and every subsequent request handler all has its own (clean) global environment. So one will get Lua exception for accessing the nil value.
原因如下:通过设计,全局环境与与之关联的Nginx请求处理程序具有完全相同的生命周期。每个请求处理程序都有自己的一组Lua全局变量,这就是请求隔离的想法。 Lua模块实际上由第一个Nginx请求处理程序加载,并由package()内置在package.loaded表中进行缓存以供以后引用,以及module()内置由一些Lua模块使用具有将全局变量设置为已加载模块表的副作用。但是这个全局变量将在请求处理程序结束时被清除,并且每个后续请求处理程序都有自己的(干净的)全局环境。因此,访问nil值会得到Lua异常

上一篇下一篇

猜你喜欢

热点阅读