VictoriaMetrics vmctl remote-read 模式：基于 Prometheus Remote Read 协议的数据迁移指南-尧图网站开发

VictoriaMetrics vmctl remote-read 模式基于 Prometheus Remote Read 协议的数据迁移指南【免费下载链接】VictoriaMetricsVictoriaMetrics: fast, cost-effective monitoring solution and time series database项目地址: https://gitcode.com/GitHub_Trending/vi/VictoriaMetrics导读本文介绍 VictoriaMetrics 数据迁移工具 vmctl 的remote-read迁移模式。该模式允许通过 Prometheus Remote Read API 从 Cortex、Mimir、Promscale、Thanos 等远程时序数据库读取历史数据并写入 VictoriaMetrics适用于在无法直接访问对象存储或快照文件场景下的数据迁移。读完本文你将掌握vmctl remote-read的完整命令用法、时间与标签过滤策略、两种协议实现SAMPLES与STREAMED_XOR_CHUNKS的取舍、以及迁移性能调优与监控手段并能结合源码理解其底层调用链。概述remote-read 模式能做什么vmctl remote-read通过 Prometheus 定义的 Remote Read API 从支持该协议的远端时序数据库拉取时间序列数据。官方文档明确支持以下系统CortexMimirPromscaleThanos通过其 remote read 协议入口在 vmctl 主文档中remote-read与prometheus快照迁移、influx、opentsdb、thanos、mimir、vm-native等并列是 vmctl 提供的七大迁移模式之一。与其他模式相比remote-read 的最大特点是完全通过 API 拉取数据不依赖源端快照文件或对象存储访问权限但代价是这是一项对资源消耗较高的操作——文档特别提示对于 Thanos 这类系统通过 remote read 协议迁移可能会较慢或在资源上较为昂贵。Remote Read API 存在两种实现方式模式协议类型特点SAMPLES默认普通批量响应兼容性最广几乎所有支持 remote read 的系统都能使用STREAMED_XOR_CHUNKS流式分块响应效率更高、对源端资源占用更少但只有 Mimir 等少数数据库支持流式版本streamed remote read效率更高但整体采用率较低这一点在迁移规划时需要权衡。快速开始本地 Prometheus 迁移示例vmctl 的下载与构建方式见 vmctl 主文档可通过make vmctl从本仓库根目录构建开发版二进制放入bin目录生产版使用make vmctl-prod。一个针对本地 Prometheus 安装的最简迁移命令如下./vmctl remote-read \ --remote-read-src-addrhttp://prometheus:9091 \ --remote-read-filter-time-start2021-10-18T00:00:00Z \ --remote-read-step-intervalhour \ --vm-addrhttp://victoria-metrics:8428 \参数说明--remote-read-src-addr源端 remote read 服务地址。vmctl 默认会在此基础上自动拼接/api/v1/read路径见下文源码级原理因此这里通常填写服务的基础地址即可。--remote-read-filter-time-start迁移数据的起始时间RFC3339 格式。--remote-read-step-interval将迁移时间范围切分为多个子区间的步长这里取hour表示按小时分片请求。--vm-addrVictoriaMetrics 目标地址。单机版对应--httpListenAddr集群版为 vminsert 地址默认http://localhost:8428。vmctl 会通过/health端点对目标做就绪检查更多说明见配置 VictoriaMetrics。注意vmctl 默认在开始导入前会弹出确认提示可通过-ssilent标志跳过交互确认--disable-progress-bar可关闭导入进度条。默认--remote-read-concurrency为 1即串行拉取。过滤按时间与标签缩小迁移范围时间过滤通过--remote-read-filter-time-start和--remote-read-filter-time-end两个标志以 RFC3339 格式例如2020-01-01T20:07:00Z限定时间范围。若未指定--remote-read-filter-time-end在 remoteread.go 处理器中可以看到 vmctl 会以当前时间自动填充结束时间if rrp.filter.timeEnd nil { t : time.Now().In(rrp.filter.timeStart.Location()) rrp.filter.timeEnd t }标签过滤标签过滤通过--remote-read-filter-label与--remote-read-filter-label-value一对标志实现。例如./vmctl remote-read \ --remote-read-filter-labeltenant \ --remote-read-filter-label-valueteam-eu \ ...将只选择带tenantteam-eu标签值对的时间序列。多重过滤v1.129.0从 v1.129.0 起过滤标志可以多次指定以进一步收窄待迁移的时间序列集合。例如同时按租户标签和指标名正则过滤./vmctl remote-read \ --remote-read-filter-labeltenant --remote-read-filter-label-valueteam-eu \ --remote-read-filter-label__name__ --remote-read-filter-label-valuecpu_.*上述命令只会迁移同时满足tenantteam-eu且指标名匹配cpu_.*正则的时间序列。需要说明的是--remote-read-filter-label-value的值本质上是正则表达式RE 类型匹配器。从客户端实现可以看到每对 label 名/值都会被构造成prompb.LabelMatcher{Type: prompb.LabelMatcher_RE}最终封装进 remote read 的查询请求中如果标签名与标签值的数量不一致NewClient会直接报错返回防止误配置。若用户完全不提供任何过滤标志main.go 中会填充一组向后兼容的默认值__name__~.*即匹配所有指标名保证行为与早期版本一致。配置详解超时、分片步长与流式模式增大 HTTP 超时避免迁移中断迁移大数据量时remote read 客户端可能达到请求超时。源码中的默认读超时为 5 分钟见 remoteread.go 常量定义的defaultReadTimeout 5 * time.Minute而--remote-read-http-timeout标志在 flags 文档中记录的默认值是0s——即未显式设置时NewClient 会回退到 5 分钟的默认超时。当看到超时或context canceled错误时应加大该标志的值例如./vmctl remote-read \ --remote-read-src-addrhttp://source:9090 \ --remote-read-http-timeout30m \ ...用分片步长降低源端压力--remote-read-step-interval允许把导出数据切分成多个区块以减轻对源--remote-read-src-addr的压力。有效值为month, day, hour, minuteflags 文档中还包含week即month, week, day, hour, minute。以--remote-read-step-intervalmonth迁移 1 年数据为例vmctl 会从时间范围起点开始发出 12 个独立请求。这个分片逻辑由 stepper 包实现SplitDateRange按步长把[start, end]切成连续子区间其中month步长会对齐到每月 1 日以提高块传输层面的导出效率week步长为 7 天、day/hour/minute分别递增对应时间单位。若配合--remote-read-filter-time-reverse则区间顺序反转从最新数据向最旧数据迁移。从 remoteReadProcessor.run 可以看到切分完成后 vmctl 会打印确认信息Selected time range … - … will be split into N ranges according to … step. Continue?流式模式与样本重复问题--remote-read-use-stream标志决定使用SAMPLES还是STREAMED_XOR_CHUNKS模式默认使用SAMPLES。STREAMED_XOR_CHUNKS模式对源端资源消耗小得多但仅被 Mimir 等有限数据库支持。在客户端 fetch 实现中可以看到两种模式的实际区别SAMPLES模式请求头Content-Type: application/x-protobuf响应体经 snappy 解压后整体反序列化为prompb.ReadResponse再逐条时间序列回调处理流式模式请求头Content-Type: application/x-streamed-protobuf; protoprometheus.ChunkedReadResponse使用 Prometheus 的remote.NewChunkedReader逐块读取ChunkedReadResponse边读边解析无需等待完整响应体。流式模式另一层含义是响应中携带的是XOR 压缩编码的数据块chunk而非逐样本的采样点客户端需要用chunkenc.FromData解出每个样本见 parseSamples。使用--remote-read-use-streamtrue时尤其是搭配minute这样的细小步长可能观察到写入 VictoriaMetrics 的样本数比源端更多。文档指出这是由源端底层 chunk 存储结构导致的启用 VictoriaMetrics 的去重deduplication 后这些重复样本在查询和存储层面都会被最终消除。其他连接与安全相关标志除上述核心标志外remote-read还支持认证、TLS 与自定义请求头等配置与 vmctl_remote-read_flags.md 记录一致标志说明--remote-read-user/--remote-read-password源端 basic auth 认证密码可经环境变量REMOTE_READ_PASSWORD传入--remote-read-headers随每个请求发送给源端的自定义 HTTP 头多个头用^^分隔如My-Auth:foobar^^X-Extra:v2--remote-read-cert-file/--remote-read-key-file客户端侧 TLS 证书与私钥--remote-read-CA-file校验源端连接的 CA 证书默认使用系统 CA--remote-read-server-nameTLS 连接的 server name默认取自--remote-read-src-addr--remote-read-insecure-skip-verify跳过源端 TLS 证书校验默认 false--remote-read-disable-path-append禁止自动向--remote-read-src-addr追加/api/v1/read后缀TLS 传输层通过promauth.NewTLSTransport构建见 main.go 的 remote-read 命令装配。完整命令标志参考运行./vmctl remote-read --help可查看全部标志。以下为该子命令的完整输出来源vmctl_remote-read_flags.mdNAME: vmctl remote-read - Migrate time series via Prometheus remote-read protocol USAGE: vmctl remote-read [command options] OPTIONS: -s Whether to run in silent mode. If set to true no confirmation prompts will appear. (default: false) --verbose Whether to enable verbosity in logs output. (default: false) --disable-progress-bar Whether to disable progress bar during the import. (default: false) --pushmetrics.url value [ --pushmetrics.url value ] Optional URL to push metrics. --pushmetrics.interval value Interval for pushing metrics to every -pushmetrics.url (default: 10s) --pushmetrics.extraLabel value [ --pushmetrics.extraLabel value ] Extra labels to add to pushed metrics. --pushmetrics.header value [ --pushmetrics.header value ] Optional HTTP headers to add to pushed metrics. --pushmetrics.disableCompression Whether to disable compression when pushing metrics. (default: false) --remote-read-concurrency value Number of concurrently running remote read readers (default: 1) --remote-read-filter-time-start value The time filter in RFC3339 format to select timeseries with timestamp equal or higher than provided value. E.g. 2020-01-01T20:07:00Z --remote-read-filter-time-end value The time filter in RFC3339 format to select timeseries with timestamp equal or lower than provided value. E.g. 2020-01-01T20:07:00Z --remote-read-filter-label value [ --remote-read-filter-label value ] Prometheus label name to filter timeseries by. E.g. __name__ will filter timeseries by name. (default: __name__) --remote-read-filter-label-value value [ --remote-read-filter-label-value value ] Prometheus regular expression to filter label from remote-read-filter-label-value flag. (default: .*) --remote-read Use Prometheus remote read protocol (default: false) --remote-read-use-stream Defines whether to use SAMPLES or STREAMED_XOR_CHUNKS mode. By default, is uses SAMPLES mode. (default: false) --remote-read-step-interval value The time interval to split the migration into steps. Valid values are month,week,day,hour,minute. --remote-read-filter-time-reverse Whether to reverse the order of time intervals split by --remote-read-step-interval cmd-line flag. (default: false) --remote-read-src-addr value Remote read address to perform read from. --remote-read-user value Remote read username for basic auth [$REMOTE_READ_USERNAME] --remote-read-password value Remote read password for basic auth [$REMOTE_READ_PASSWORD] --remote-read-http-timeout value Timeout defines timeout for HTTP requests made by remote read client (default: 0s) --remote-read-headers value Optional HTTP headers to send with each request to the corresponding remote source storage --remote-read-cert-file value Optional path to client-side TLS certificate file to use when connecting to -remote-read-src-addr --remote-read-key-file value Optional path to client-side TLS key to use when connecting to -remote-read-src-addr --remote-read-CA-file value Optional path to TLS CA file to use for verifying connections to -remote-read-src-addr. By default, system CA is used --remote-read-server-name value Optional TLS server name to use for connections to remoteReadSrcAddr. --remote-read-insecure-skip-verify Whether to skip TLS certificate verification when connecting to the remote read address (default: false) --remote-read-disable-path-append Whether to disable automatic appending of the /api/v1/read suffix to --remote-read-src-addr (default: false) --vm-addr value VictoriaMetrics address to perform import requests. (default: http://localhost:8428) --vm-user value VictoriaMetrics username for basic auth [$VM_USERNAME] --vm-password value VictoriaMetrics password for basic auth [$VM_PASSWORD] --vm-headers value Optional HTTP headers to send with each request to the corresponding destination address. --vm-bearer-token value Optional bearer auth token to use for the corresponding --vm-addr --vm-account-id value AccountID is an arbitrary 32-bit integer identifying namespace for data ingestion (aka tenant). Required for cluster version. --vm-concurrency value Number of workers concurrently performing import requests to VM (default: 2) --vm-compress Whether to apply gzip compression to import requests (default: true) --vm-batch-size value How many samples importer collects before sending the import request to VM (default: 200000) --vm-significant-figures value The number of significant figures to leave in metric values before importing. Zero value saves all the significant figures. (default: 0) --vm-round-digits value Round metric values to the given number of decimal digits after the point. (default: 100) --vm-extra-label value [ --vm-extra-label value ] Extra labels, that will be added to imported timeseries. --vm-rate-limit value Optional data transfer rate limit in bytes per second. (default: 0) --vm-cert-file value Optional path to client-side TLS certificate file to use when connecting to --vm-addr --vm-key-file value Optional path to client-side TLS key to use when connecting to --vm-addr --vm-CA-file value Optional path to TLS CA file to use for verifying connections to --vm-addr. --vm-server-name value Optional TLS server name to use for connections to --vm-addr. --vm-insecure-skip-verify Whether to skip tls verification when connecting to --vm-addr (default: false) --vm-backoff-retries value How many import retries to perform before giving up. (default: 10) --vm-backoff-factor value Factor to multiply the base duration after each failed import retry. Must be greater than 1.0 (default: 1.8) --vm-backoff-min-duration value Minimum duration to wait before the first import retry. (default: 2s) --help, -h show help其中--vm-*系列为所有迁移模式共用的目标端VictoriaMetrics配置其核心项说明可参考 vmctl 主文档的配置与迁移建议章节集群版必须额外设置--vm-account-id并将--vm-addr指向 vminsert如http://vminsert-addr:8480--vm-concurrency决定并发写入 worker 数每个 worker 最多可占满 VictoriaMetrics 一个 vCPU 核应结合目标实例的 CPU 资源设置。源码级原理一次 remote read 迁移的完整调用链入口装配main.go在 main.go 中remote-read子命令依次完成校验--remote-read-src-addr的 URL 合法性httputil.CheckURL用promauth.NewTLSTransport构建 TLS 传输层用remoteread.NewClient创建客户端传入地址、认证、超时、流式开关、请求头、标签过滤与路径追加开关用initConfigVM构建目标端 Importer组装remoteReadProcessor并执行run。请求构造与响应解析remoteread 包remoteread/remoteread.go 是协议层面的核心实现关键点包括默认路径拼接remoteReadPath /api/v1/read请求 URL 通过url.JoinPath(c.addr, remoteReadPath)生成只有设置--remote-read-disable-path-append时才使用完整自定义地址fetch 实现。请求协议细节请求体为 protobuf 序列化并经 snappy 压缩的prompb.ReadRequest携带Content-Encoding: snappy、X-Prometheus-Remote-Read-Version: 0.1.0等头查询的EndTimestampMs被减 1 以避免与下一个时间分片重叠Read 方法。浮点样本与原生直方图双通道处理无论是SAMPLES还是流式响应一条序列可能同时包含 float 样本和原生直方图样本序列可能在某个时间点从 float 切换到原生直方图表示两者被独立解析后分别回调见processResponse/processStreamResponse。原生直方图转换convertHistograms把每个原生直方图样本转换为name_count、name_sum以及一组带vmrange标签的name_bucket非累计桶序列与 VictoriaMetrics 通过 remote write 协议接收原生直方图时的处理方式保持一致区别在于对带自定义桶NHCB的直方图边界取自自定义值而非指数公式估算。vmrange标签值由formatVmrange以科学计数法3 位小数格式化为lower...upper形式。错误处理非 200/204 的响应码会连同响应体一起作为错误返回context.Canceled被识别为请求已取消错误。分片与并发调度主处理器remoteread.go 主处理器中run方法先调用stepper.SplitDateRange生成时间分片再启动--remote-read-concurrency个 goroutine 从 channel 消费分片并调用Client.Read数据经Importer.Input批量写入 VictoriaMetrics。导入结束后会打印Import finished!与统计信息stats。测试印证remoteread_test.go 对上述行为提供了完整验证TestProcessResponse验证普通模式下 float 序列与直方图序列混合响应被正确拆分为 7 条序列1 条 float _count/_sum 4 条_bucketTestProcessStreamResponse验证流式模式下 XOR、直方图以及UNKNOWN未设置 type 的 XOR三种 chunk 的解析共得到 8 条序列TestConvertHistograms与TestParseHistograms分别覆盖直方图转换与 chunk 解析的边界情况如空样本、缺指标名、不支持编码报错。迁移性能监控vmctl 自 v1.135.0 起支持把内部指标推送到远端存储用于监控迁移进度与性能详见 vmctl 主文档。remote-read 模式相关的指标为指标含义vmctl_remote_read_migration_ranges_total本次迁移计划处理的总时间分片数vmctl_remote_read_migration_ranges_processed已处理完成的时间分片数vmctl_remote_read_migration_errors_total迁移过程中发生的错误累计数这些指标定义于主处理器每个分片成功/失败时都会递增。配合--pushmetrics.url即可推送例如./vmctl remote-read \ --remote-read-src-addrhttp://source:9090 \ --remote-read-filter-time-start2024-01-01T00:00:00Z \ --vm-addrhttp://localhost:8428 \ --pushmetrics.urlhttp://localhost:8428/api/v1/import/prometheus \ --pushmetrics.extraLabeljobvmctl \ --pushmetrics.extraLabelinstancemigration-1迁移完成百分比可用 PromQL 计算vmctl_remote_read_migration_ranges_processed / vmctl_remote_read_migration_ranges_total * 100。迁移建议与注意事项综合 vmctl 主文档的 migration tips 与 remote-read 文档实践中有几点值得注意迁移速度的三要素网络带宽vmctl 充当源与目标之间的代理、源端返回数据的速度、目标端接收存储数据的速度。可依据实际情况调整--remote-read-concurrency源端读取并发与--vm-concurrency目标端写入并发。警惕超时与context canceled大数据量下优先调大--remote-read-http-timeout。流式模式的重复样本使用--remote-read-use-streamtrue且步长较小时可能写入更多样本建议在目标端启用去重以最终消除重复。合理设置分片步长--remote-read-step-interval越小单次请求对源端压力越小但请求次数越多month步长还会对齐到自然月以提升块传输效率。迁移是回填过程参考 VictoriaMetrics 的 Backfilling 相关建议vmctl 本身不提供 relabel 能力标签改写应使用 VictoriaMetrics 的 relabeling 机制。压缩优化对聚合类指标如avg、rate结果可用--vm-significant-figures或--vm-round-digits减少有效位数以提升磁盘压缩率默认--vm-round-digits100表示基本不做舍入显式设置 2 可将1.2345舍入为1.23。延伸阅读vmctl 主文档所有迁移模式的总览、目标端配置、Importer 统计信息解读、限速、额外标签与监控Cortex 迁移、Mimir 迁移、Promscale 迁移、Thanos 迁移含 remote read 协议入口完整标志列表vmctl_remote-read_flags.md相关源码命令装配、主处理器、协议客户端、时间分片 stepper、协议测试【免费下载链接】VictoriaMetricsVictoriaMetrics: fast, cost-effective monitoring solution and time series database项目地址: https://gitcode.com/GitHub_Trending/vi/VictoriaMetrics创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考

建站资讯详情

VictoriaMetrics vmctl remote-read 模式：基于 Prometheus Remote Read 协议的数据迁移指南