Skip to content

Apache Thrift 协议测试

The Apache Thrift software framework, for scalable cross-language services development

Apache Thrift 软件框架，用于可扩展的跨语言服务开发

Apache Thrift

The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.

Apache Thrift 软件框架，用于可扩展的跨语言服务开发，将软件堆栈与代码生成引擎相结合，以构建在 C++、Java、Python、PHP、Ruby、Erlang、Perl、Haskell、C#、 Cocoa、JavaScript、Node.js、Smalltalk、OCaml 和 Delphi 等语言。

thrift 数据类型定义与服务接口定义示例

/**
 * Ahh, now onto the cool part, defining a service. Services just need a name
 * and can optionally inherit from another service using the extends keyword.
 */
service Calculator extends shared.SharedService {

  /**
   * A method definition looks like C code. It has a return type, arguments,
   * and optionally a list of exceptions that it may throw. Note that argument
   * lists and exception lists are specified using the exact same syntax as
   * field lists in struct or exception definitions.
   */

   void ping(),

   i32 add(1:i32 num1, 2:i32 num2),

   i32 calculate(1:i32 logid, 2:Work w) throws (1:InvalidOperation ouch),

   /**
    * This method has a oneway modifier. That means the client only makes
    * a request and does not listen for any response at all. Oneway methods
    * must be void.
    */
   oneway void zip()

}

def main():
    # Make socket
    transport = TSocket.TSocket('localhost', 9090)

    # Buffering is critical. Raw sockets are very slow
    transport = TTransport.TBufferedTransport(transport)

    # Wrap in a protocol
    protocol = TBinaryProtocol.TBinaryProtocol(transport)

    # Create a client to use the protocol encoder
    client = Calculator.Client(protocol)

    # Connect!
    transport.open()

    client.ping()
    print('ping()')

    sum_ = client.add(1, 1)

Thrift 协议优点

  • 紧凑的数据存储
  • 快速解析
  • 支持众多编程语言
  • 与 pb 类似，pb 属于 google，thrift 属于更加开放的 apache 基金会

thrift 的技术架构

  • transport
  • protocol
  • server client
  • processor

使用流程总结

  • 定义消息 Message
  • 编译消息到各个语言代码 Compiler
  • 在各个语言中构建 Runtime
  • 在项目中应用 序列化 反序列化 网络传输等

服务端与用户端流程

  • 服务端流程：server + protocol + processor
  • 用户端流程：transport + protocol + client

Thrift 协议规范

thrift IDL 接口定义语法

/**
 * Thrift lets you do typedefs to get pretty names for your types. Standard
 * C style here.
 */
typedef i32 MyInteger

/**
 * Thrift also lets you define constants for use across languages. Complex
 * types and structs are specified using JSON notation.
 */
const i32 INT32CONSTANT = 9853
const map<string,string> MAPCONSTANT = {'hello':'world', 'goodnight':'moon'}

/**
 * You can define enums, which are just 32 bit integers. Values are optional
 * and start at 1 if not supplied, C style again.
 */
enum Operation {
  ADD = 1,
  SUBTRACT = 2,
  MULTIPLY = 3,
  DIVIDE = 4
}

/**
 * Structs are the basic complex data structures. They are comprised of fields
 * which each have an integer identifier, a type, a symbolic name, and an
 * optional default value.
 *
 * Fields can be declared "optional", which ensures they will not be included
 * in the serialized output if they aren't set.  Note that this requires some
 * manual management in some languages.
 */
struct Work {
  1: i32 num1 = 0,
  2: i32 num2,
  3: Operation op,
  4: optional string comment,
}

/**
 * Structs can also be exceptions, if they are nasty.
 */
exception InvalidOperation {
  1: i32 whatOp,
  2: string why
}

/**
 * Ahh, now onto the cool part, defining a service. Services just need a name
 * and can optionally inherit from another service using the extends keyword.
 */
service Calculator extends shared.SharedService {

  /**
   * A method definition looks like C code. It has a return type, arguments,
   * and optionally a list of exceptions that it may throw. Note that argument
   * lists and exception lists are specified using the exact same syntax as
   * field lists in struct or exception definitions.
   */

   void ping(),

   i32 add(1:i32 num1, 2:i32 num2),

   i32 calculate(1:i32 logid, 2:Work w) throws (1:InvalidOperation ouch),

   /**
    * This method has a oneway modifier. That means the client only makes
    * a request and does not listen for any response at all. Oneway methods
    * must be void.
    */
   oneway void zip()

}

thrift 主要定义语法

  • 原始类型： integers booleans floats string
  • 类型： struct containers service
  • 字段规则： required optional

协议编译

编译器下载

编译器命令行用法

thrift
Usage: thrift [options] file

thrift -r --gen py -o python tutorial.thrift

thrift -r --gen java -o java tutorial.thrift

Python 应用示例

编译 Python 代码

thrift -r --gen py -o python tutorial.thrift

安装依赖

pip install thrift

使用 Python 构建服务

```Python{data-line-numbers="12-100"}

sys.path.append('gen-py')

sys.path.insert(0, glob.glob('../../lib/py/build/lib*')[0])

from demo.calculator_handler import CalculatorHandler from tutorial import Calculator

from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from thrift.server import TServer

if name == 'main': handler = CalculatorHandler() processor = Calculator.Processor(handler) transport = TSocket.TServerSocket(host='127.0.0.1', port=9090) tfactory = TTransport.TBufferedTransportFactory() pfactory = TBinaryProtocol.TBinaryProtocolFactory()

server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)

# You could do one of these for a multithreaded server
# server = TServer.TThreadedServer(
#     processor, transport, tfactory, pfactory)
# server = TServer.TThreadPoolServer(
#     processor, transport, tfactory, pfactory)

print('Starting the server...')
server.serve()
print('done.')

```

Python 测试代码

Python{data-line-numbers="4-100"} class TestThrift: def setup_class(self): # Make socket self.transport = TSocket.TSocket('localhost', 9090) # Buffering is critical. Raw sockets are very slow self.transport = TTransport.TBufferedTransport(self.transport) # Wrap in a protocol protocol = TBinaryProtocol.TBinaryProtocol(self.transport) # Create a client to use the protocol encoder self.client = Calculator.Client(protocol) # Connect! self.transport.open() def teardown_class(self): # Close! self.transport.close() def test_ping(self): self.client.ping() print('ping()') def test_add(self): sum_ = self.client.add(1, 1) print('1+1=%d' % sum_) assert 2 == sum_ def test_divide(self): work = Work() work.op = Operation.DIVIDE work.num1 = 1 work.num2 = 0 with pytest.raises(InvalidOperation): quotient = self.client.calculate(1, work) def test_subtract(self): work = Work() work.op = Operation.SUBTRACT work.num1 = 15 work.num2 = 10 diff = self.client.calculate(1, work) print('15-10=%d' % diff) assert 5 == diff

Java 应用案例

编译 Java 代码

thrift -r --gen java -o java tutorial.thrift

安装 Runtime 依赖

    <dependencies>
        <dependency>
            <groupId>org.apache.thrift</groupId>
            <artifactId>libthrift</artifactId>
            <version>LATEST</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>5.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.annotation</groupId>
            <artifactId>javax.annotation-api</artifactId>
            <version>LATEST</version>
        </dependency>
    </dependencies>

使用 Java 构建服务

```java{data-line-numbers="41-100"} / * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding copyright ownership. The ASF licenses this file * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, * software distributed under the License is distributed on an * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY * KIND, either express or implied. See the License for the * specific language governing permissions and limitations * under the License. /

import org.apache.thrift.server.TServer; import org.apache.thrift.server.TServer.Args; import org.apache.thrift.server.TSimpleServer; import org.apache.thrift.server.TThreadPoolServer; import org.apache.thrift.transport.TSSLTransportFactory; import org.apache.thrift.transport.TServerSocket; import org.apache.thrift.transport.TServerTransport; import org.apache.thrift.transport.TSSLTransportFactory.TSSLTransportParameters;

// Generated code import tutorial.; import shared.;

import java.util.HashMap;

public class JavaServer {

public static CalculatorHandler handler;

public static Calculator.Processor processor;

public static void main(String[] args) {
    try {
        handler = new CalculatorHandler();
        processor = new Calculator.Processor(handler);

        Runnable simple = new Runnable() {
            public void run() {
                simple(processor);
            }
        };
        new Thread(simple).start();
    } catch (Exception x) {
        x.printStackTrace();
    }
}

public static void simple(Calculator.Processor processor) {
    try {
        TServerTransport serverTransport = new TServerSocket(9090);
        TServer server = new TSimpleServer(new Args(serverTransport).processor(processor));

        // Use this for a multithreaded server
        // TServer server = new TThreadPoolServer(new TThreadPoolServer.Args(serverTransport).processor(processor));

        System.out.println("Starting the simple server...");
        server.serve();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

public static void secure(Calculator.Processor processor) {
    try {
        /*
         * Use TSSLTransportParameters to setup the required SSL parameters. In this example
         * we are setting the keystore and the keystore password. Other things like algorithms,
         * cipher suites, client auth etc can be set.
         */
        TSSLTransportParameters params = new TSSLTransportParameters();
        // The Keystore contains the private key
        params.setKeyStore("../../lib/java/test/.keystore", "thrift", null, null);

        /*
         * Use any of the TSSLTransportFactory to get a server transport with the appropriate
         * SSL configuration. You can use the default settings if properties are set in the command line.
         * Ex: -Djavax.net.ssl.keyStore=.keystore and -Djavax.net.ssl.keyStorePassword=thrift
         *
         * Note: You need not explicitly call open(). The underlying server socket is bound on return
         * from the factory class.
         */
        TServerTransport serverTransport = TSSLTransportFactory.getServerSocket(9091, 0, null, params);
        TServer server = new TSimpleServer(new Args(serverTransport).processor(processor));

        // Use this for a multi threaded server
        // TServer server = new TThreadPoolServer(new TThreadPoolServer.Args(serverTransport).processor(processor));

        System.out.println("Starting the secure server...");
        server.serve();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

} ```

Java 测试代码

java{data-line-numbers="44-100"} /* * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding copyright ownership. The ASF licenses this file * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, * software distributed under the License is distributed on an * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY * KIND, either express or implied. See the License for the * specific language governing permissions and limitations * under the License. */ // Generated code import org.apache.thrift.transport.TTransportException; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import tutorial.*; import shared.*; import org.apache.thrift.TException; import org.apache.thrift.transport.TSSLTransportFactory; import org.apache.thrift.transport.TTransport; import org.apache.thrift.transport.TSocket; import org.apache.thrift.transport.TSSLTransportFactory.TSSLTransportParameters; import org.apache.thrift.protocol.TBinaryProtocol; import org.apache.thrift.protocol.TProtocol; import static org.junit.jupiter.api.Assertions.*; public class JavaClientTest { static TTransport transport; static Calculator.Client client; @BeforeAll static void beforeAll() throws TTransportException { transport = new TSocket("localhost", 9090); transport.open(); TProtocol protocol = new TBinaryProtocol(transport); client = new Calculator.Client(protocol); } @AfterAll static void afterAll() { transport.close(); } @Test void add() throws TException { client.ping(); int sum = client.add(1, 1); assertEquals(2, sum); } @Test void divide() throws TException { Work work = new Work(); work.op = Operation.DIVIDE; work.num1 = 1; work.num2 = 0; assertThrows(TException.class, () -> { int quotient = client.calculate(1, work); }); } @Test void subtract() throws TException { Work work = new Work(); work.op = Operation.SUBTRACT; work.num1 = 15; work.num2 = 10; int diff = client.calculate(1, work); assertEquals(5, diff); } }

Q&A