技术文程序员技术干货

Protocol Buffers简明教程

2017-03-19  本文已影响0人  ginobefun

随着微服务架构的流行，RPC框架渐渐地成为服务框架的一个重要部分。在很多RPC的设计中，都采用了高性能的编解码技术，Protocol Buffers就属于其中的佼佼者。Protocol Buffers是Google开源的一个语言无关、平台无关的通信协议，其小巧、高效和友好的兼容性设计，使其被广泛使用。

概述

protobuf是什么？

Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

为什么叫“Protocol Buffers”？

官方如是说：

The name originates from the early days of the format, before we had the protocol buffer compiler to generate classes for us. At the time, there was a class called ProtocolBuffer which actually acted as a buffer for an individual method. Users would add tag/value pairs to this buffer individually by calling methods like AddValue(tag, value). The raw bytes were stored in a buffer which could then be written out once the message had been constructed.

Since that time, the "buffers" part of the name has lost its meaning, but it is still the name we use. Today, people usually use the term "protocol message" to refer to a message in an abstract sense, "protocol buffer" to refer to a serialized copy of a message, and "protocol message object" to refer to an in-memory object representing the parsed message.

核心特点

“变态的”性能表现

有位网友曾经做过各种通用序列化协议技术的对比，我这里直接拿来给大家感受一下：

序列化响应时间对比

序列化响应时间对比

序列化bytes对比

序列化bytes对比

具体的数字

具体的数字

快速开始

以下示例源码已上传至github：https://github.com/ginobefun/learning_projects/tree/master/learning-protobuf

新建一个maven项目并添加依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.ginobefunny.learning</groupId>
    <artifactId>leanring-protobuf</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.2.0</version>
        </dependency>
    </dependencies>
</project>

新建protobuf的消息定义文件addressbook.proto

syntax = "proto3"; // 声明为protobuf 3定义文件
package tutorial;

option java_package = "com.ginobefunny.learning.protobuf.message"; // 声明生成消息类的java包路径
option java_outer_classname = "AddressBookProtos";  // 声明生成消息类的类名

message Person {
  string name = 1;
  int32 id = 2;
  string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
  }

  repeated PhoneNumber phones = 4;
}

message AddressBook {
  repeated Person people = 1;
}

使用protoc工具生成消息对应的Java类

protoc -I=. --java_out=src/main/java addressbook.proto

编写测试类写入和读取序列化文件

public class AddPerson {

    // 通过用户输入构建一个Person对象
    static AddressBookProtos.Person promptForAddress(BufferedReader stdin,
                                                     PrintStream stdout) throws IOException {
        AddressBookProtos.Person.Builder person = AddressBookProtos.Person.newBuilder();

        stdout.print("Enter person ID: ");
        person.setId(Integer.valueOf(stdin.readLine()));

        stdout.print("Enter name: ");
        person.setName(stdin.readLine());

        stdout.print("Enter email address (blank for none): ");
        String email = stdin.readLine();
        if (email.length() > 0) {
            person.setEmail(email);
        }

        while (true) {
            stdout.print("Enter a phone number (or leave blank to finish): ");
            String number = stdin.readLine();
            if (number.length() == 0) {
                break;
            }

            AddressBookProtos.Person.PhoneNumber.Builder phoneNumber =
                    AddressBookProtos.Person.PhoneNumber.newBuilder().setNumber(number);

            stdout.print("Is this a mobile, home, or work phone? ");
            String type = stdin.readLine();
            if (type.equals("mobile")) {
                phoneNumber.setType(AddressBookProtos.Person.PhoneType.MOBILE);
            } else if (type.equals("home")) {
                phoneNumber.setType(AddressBookProtos.Person.PhoneType.HOME);
            } else if (type.equals("work")) {
                phoneNumber.setType(AddressBookProtos.Person.PhoneType.WORK);
            } else {
                stdout.println("Unknown phone type.  Using default.");
            }

            person.addPhones(phoneNumber);
        }

        return person.build();
    }

    // 加载指定的序列化文件（如不存在则创建一个新的），再通过用户输入增加一个新的联系人到地址簿，最后序列化到文件中
    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            System.err.println("Usage:  AddPerson ADDRESS_BOOK_FILE");
            System.exit(-1);
        }

        AddressBookProtos.AddressBook.Builder addressBook = AddressBookProtos.AddressBook.newBuilder();

        // Read the existing address book.
        try {
            addressBook.mergeFrom(new FileInputStream(args[0]));
        } catch (FileNotFoundException e) {
            System.out.println(args[0] + ": File not found.  Creating a new file.");
        }

        // Add an address.
        addressBook.addPeople(promptForAddress(new BufferedReader(new InputStreamReader(System.in)),
                        System.out));

        // Write the new address book back to disk.
        FileOutputStream output = new FileOutputStream(args[0]);
        addressBook.build().writeTo(output);
        output.close();
    }
}

public class ListPeople {

    // 打印地址簿中所有联系人信息
    static void print(AddressBookProtos.AddressBook addressBook) {
        for (AddressBookProtos.Person person: addressBook.getPeopleList()) {
            System.out.println("Person ID: " + person.getId());
            System.out.println("  Name: " + person.getName());
            if (!person.getPhonesList().isEmpty()) {
                System.out.println("  E-mail address: " + person.getEmail());
            }

            for (AddressBookProtos.Person.PhoneNumber phoneNumber : person.getPhonesList()) {
                switch (phoneNumber.getType()) {
                    case MOBILE:
                        System.out.print("  Mobile phone #: ");
                        break;
                    case HOME:
                        System.out.print("  Home phone #: ");
                        break;
                    case WORK:
                        System.out.print("  Work phone #: ");
                        break;
                }
                System.out.println(phoneNumber.getNumber());
            }
        }
    }

    // 加载指定的序列化文件，并输出所有联系人信息
    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            System.err.println("Usage:  ListPeople ADDRESS_BOOK_FILE");
            System.exit(-1);
        }

        // Read the existing address book.
        AddressBookProtos.AddressBook addressBook =
                AddressBookProtos.AddressBook.parseFrom(new FileInputStream(args[0]));

        print(addressBook);
    }
}

验证效果

先添加一个联系人Gino

添加一个联系人Gino

再添加一个联系人Slightly

添加一个联系人Gino

最后显示所有联系人信息

添加一个联系人Gino

实例小结

深入学习

关于proto文件

protobuf版本

message结构

字段描述符

字段描述符用于描述字段出现的频率，有以下两个可选值：

字段类型

message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
  enum Corpus {
    UNIVERSAL = 0;
    WEB = 1;
    IMAGES = 2;
    LOCAL = 3;
    NEWS = 4;
    PRODUCTS = 5;
    VIDEO = 6;
  }
  Corpus corpus = 4;
}

字段对应的Tag

字段的默认值

protobuf 2.X版本是支持在字段中声明默认值的，但是在3.X版本中去掉了默认值的定义，主要是为了区别用户是否设置了一个和默认值一样的值的情况。对于3.X版本，protobuf采用以下规则处理默认值：

Map字段类型

map<string, Project> projects = 3;

导入其他proto文件

import "myproject/other_protos.proto";

如果proto中的message要扩展怎么办？

proto具有很好的扩展性，但是也要遵循以下原则：

Any消息类型

Oneof关键字

message LoginMessage {
  oneof user_identifier {
    string user_name = 4;
    string phone_num = 5;
    string user_email = 6;
  }
  
  string password = 10;
}

定义服务

service SearchService {
  rpc Search (SearchRequest) returns (SearchResponse);
}

小结

参考资料

扫一扫 关注我的微信公众号
上一篇下一篇

猜你喜欢

热点阅读