feat: add metricsServer for health checks, metrics and pprof (#5578)

Add a separate HTTP listener for observability, configured through
metricsServer in both frps and frpc. It serves /healthz and, when
pprofEnable is set, the pprof handlers except /debug/pprof/cmdline. It
never serves the dashboard, static assets or the admin API, and it does
not touch the global assets filesystem.

frps also serves /metrics on this listener with both runtime and business
metrics, enabled by metricsServer.port alone. The in-memory collector
still starts only with the dashboard, and the Prometheus collector starts
either with metricsServer or with enablePrometheus plus webServer, so a
configuration that collected nothing before still collects nothing.

The /healthz, /metrics and pprof endpoints on webServer, and
enablePrometheus, keep working but are deprecated and log a hint at
startup. frpc does not serve /metrics yet.
This commit is contained in:
fatedier
2026-10-19 20:08:39 +08:00
committed by GitHub
parent 00e4fc4dd3
commit f13fdd58d4
20 changed files with 858 additions and 27 deletions
+22 -4
View File
@@ -616,13 +616,31 @@ Proxies and visitors managed through the Store are saved to disk and automatical
When web server is enabled, frps will save monitor data in cache for 7 days. It will be cleared after process restart.
Prometheus is also supported.
Both frps and frpc support a separate `metricsServer` listener for health checks and optional profiling, without enabling the dashboard or admin API. It is disabled by default (`metricsServer.port = 0`) and binds to `127.0.0.1` unless configured otherwise.
`/healthz` returns an empty HTTP 200 response without authentication. It checks that the HTTP server is responding; it does not indicate whether frpc is connected to frps or whether proxies are ready.
#### Prometheus
Enable dashboard first, then configure `enablePrometheus = true` in `frps.toml`.
Configure a dedicated listener in `frps.toml`:
`http://{dashboard_addr}/metrics` will provide prometheus monitor data.
```toml
metricsServer.addr = "127.0.0.1"
metricsServer.port = 7501
metricsServer.user = "monitor"
metricsServer.password = "change-me"
# Optional TLS, independent of webServer:
# metricsServer.tls.certFile = "metrics.crt"
# metricsServer.tls.keyFile = "metrics.key"
# Profiling is disabled unless explicitly enabled:
# metricsServer.pprofEnable = true
```
Scrape `http://127.0.0.1:7501/metrics` using these Basic Auth credentials. This automatically enables frps business metrics, even when `webServer` is disabled and `enablePrometheus` is unset. Credentials and TLS are independent of `webServer`; when credentials are configured, both `/metrics` and profiling endpoints require authentication. `/healthz` remains anonymous.
Set `metricsServer.pprofEnable = true` to expose `/debug/pprof/` and profiling endpoints on this listener. `/debug/pprof/cmdline` is excluded. frpc supports the same health and profiling settings, but does not provide `/metrics` yet.
The `/healthz`, `/metrics`, and pprof endpoints on `webServer` are deprecated but remain available. `enablePrometheus = true` still enables the legacy frps `/metrics` endpoint on the dashboard listener. Migrate health probes, Prometheus scrapes, and profiling to `metricsServer`; the dashboard and admin APIs continue to use `webServer`.
### Authenticating the Client
@@ -714,7 +732,7 @@ transport.tls.trustedCaFile = "ca.crt"
By default, when neither `transport.tls.trustedCaFile` nor `transport.tls.useSystemCa` is set, `frpc` does not verify the server certificate. Set `transport.tls.useSystemCa = true` to verify it using the host's system CAs. If `trustedCaFile` is also set, its CA certificates are added to the system CA pool; setting only `trustedCaFile` trusts only the CAs in that file. CA loading or certificate verification failures cause the connection to fail without falling back to another trust source or skipping verification.
`useSystemCa` is supported only in `frpc`'s `transport.tls`, and requires `transport.tls.enable = true` unless `transport.protocol = "wss"`. With `wss`, the outer TLS connection is always enabled and verification applies to the TLS endpoint, which may be a reverse proxy. Setting `useSystemCa = true` in `frps`'s `transport.tls` or either process's `webServer.tls` is a configuration error.
`useSystemCa` is supported only in `frpc`'s `transport.tls`, and requires `transport.tls.enable = true` unless `transport.protocol = "wss"`. With `wss`, the outer TLS connection is always enabled and verification applies to the TLS endpoint, which may be a reverse proxy. Setting `useSystemCa = true` in `frps`'s `transport.tls` or either process's `webServer.tls` or `metricsServer.tls` is a configuration error.
**`frps` TLS settings:**
+3
View File
@@ -2,6 +2,9 @@
* frpc can verify server certificates using system CAs, optionally combined with CAs from `trustedCaFile`.
* `frpc reload` now refreshes certificate files for unchanged `https2http` and `https2https` proxies without interrupting existing connections. Failed refreshes keep the previous certificate and report the error in proxy status and logs while reload still succeeds.
* Added `metricsServer` to both frps and frpc, a separate listener for health checks and optional profiling that does not expose the dashboard or the admin API. It is disabled unless `metricsServer.port` is set, defaults to `127.0.0.1`, and has its own credentials and TLS. In frps, setting the port also serves `/metrics` with both runtime and business metrics, so `enablePrometheus` is not needed there.
The `/healthz`, `/metrics` and pprof endpoints on `webServer`, and `enablePrometheus`, still work but are deprecated: configure `metricsServer` instead. They will be removed in a later release.
## Fixes
+91
View File
@@ -0,0 +1,91 @@
// Copyright 2026 The frp Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package client
import (
"context"
"net"
"net/http"
"path/filepath"
"strconv"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/fatedier/frp/pkg/config/source"
v1 "github.com/fatedier/frp/pkg/config/v1"
)
func TestNewServiceReleasesWebServerOnMetricsServerFailure(t *testing.T) {
for _, mode := range []string{"port conflict", "invalid TLS certificate"} {
t.Run(mode, func(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer ln.Close()
cfg := &v1.ClientCommonConfig{
WebServer: v1.WebServerConfig{Port: getFreeTCPPort(t)},
MetricsServer: v1.MetricsServerConfig{Port: ln.Addr().(*net.TCPAddr).Port},
}
if mode == "invalid TLS certificate" {
cfg.MetricsServer.TLS = &v1.TLSConfig{
CertFile: filepath.Join(t.TempDir(), "missing.crt"), KeyFile: "missing.key",
}
require.NoError(t, ln.Close())
}
svr, err := NewService(ServiceOptions{
Common: cfg, ConfigSourceAggregator: source.NewAggregator(source.NewConfigSource()),
})
if svr != nil {
svr.Close()
}
require.Error(t, err)
require.Nil(t, svr)
assertLifecycleAdminPort(t, net.JoinHostPort(cfg.WebServer.Addr, strconv.Itoa(cfg.WebServer.Port)), true)
if mode == "invalid TLS certificate" {
assertLifecycleAdminPort(t, ln.Addr().String(), true)
}
})
}
}
func TestServiceMetricsServerLifecycle(t *testing.T) {
svr, err := NewService(ServiceOptions{
Common: &v1.ClientCommonConfig{
LoginFailExit: new(false), MetricsServer: v1.MetricsServerConfig{Port: getFreeTCPPort(t)},
},
ConfigSourceAggregator: source.NewAggregator(source.NewConfigSource()),
ConnectorCreator: func(context.Context, *v1.ClientCommonConfig) Connector {
return &failingConnector{err: net.ErrClosed}
},
})
require.NoError(t, err)
require.Nil(t, svr.webServer)
address := svr.metricsServer.Address()
done := make(chan struct{})
go func() { _ = svr.Run(context.Background()); close(done) }()
defer func() {
svr.Close()
waitLifecycleSignal(t, done)
assertLifecycleAdminPort(t, address, true)
}()
client := &http.Client{Timeout: 3 * time.Second}
for path, status := range map[string]int{"/healthz": http.StatusOK, "/metrics": http.StatusNotFound} {
resp, err := client.Get("http://" + address + path)
require.NoError(t, err)
require.Equal(t, status, resp.StatusCode)
require.NoError(t, resp.Body.Close())
}
}
+31 -2
View File
@@ -124,6 +124,7 @@ type Service struct {
// web server for admin UI and apis
webServer *httppkg.Server
metricsServer *httppkg.Server
vnetController *vnet.Controller
@@ -188,8 +189,8 @@ func NewService(options ServiceOptions) (*Service, error) {
proxyCfgs = config.CompleteProxyConfigurers(proxyCfgs)
visitorCfgs = config.CompleteVisitorConfigurers(visitorCfgs)
// Create the web server after all fallible steps so its listener is not
// leaked when an earlier error causes NewService to return.
// Create HTTP servers after all other fallible steps, rolling back the web
// listener if the metrics server cannot be created.
var webServer *httppkg.Server
if options.Common.WebServer.Port > 0 {
ws, err := httppkg.NewServer(options.Common.WebServer)
@@ -198,11 +199,19 @@ func NewService(options ServiceOptions) (*Service, error) {
}
webServer = ws
}
metricsServer, err := httppkg.NewMetricsServer(options.Common.MetricsServer, nil)
if err != nil {
if webServer != nil {
_ = webServer.Close()
}
return nil, err
}
s := &Service{
ctx: context.Background(),
auth: authRuntime,
webServer: webServer,
metricsServer: metricsServer,
common: options.Common,
reloadCommon: options.Common,
configFilePath: options.ConfigFilePath,
@@ -263,6 +272,12 @@ func (svr *Service) Run(ctx context.Context) error {
}
if svr.webServer != nil {
if svr.common.WebServer.PprofEnable {
log.Warnf("webServer pprof endpoints are deprecated; use metricsServer.pprofEnable instead")
}
if svr.common.MetricsServer.Port == 0 {
log.Infof("webServer /healthz is deprecated; health probes can migrate to metricsServer.port")
}
webServer := svr.webServer
go func() {
log.Infof("admin server listen on %s", webServer.Address())
@@ -272,6 +287,16 @@ func (svr *Service) Run(ctx context.Context) error {
}()
}
if svr.metricsServer != nil {
metricsServer := svr.metricsServer
go func() {
log.Infof("metrics server listen on %s", metricsServer.Address())
if err := metricsServer.Run(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Warnf("metrics server exit with error: %v", err)
}
}()
}
// first login to frps
svr.loopLoginUntilSuccess(10*time.Second, lo.FromPtr(svr.common.LoginFailExit))
if svr.ctl == nil {
@@ -497,6 +522,10 @@ func (svr *Service) stop() {
svr.webServer.Close()
svr.webServer = nil
}
if svr.metricsServer != nil {
svr.metricsServer.Close()
svr.metricsServer = nil
}
if svr.vnetController != nil {
_ = svr.vnetController.Stop()
svr.vnetController = nil
+13
View File
@@ -79,9 +79,22 @@ webServer.password = "admin"
# Admin assets directory. By default, these assets are bundled with frpc.
# webServer.assetsDir = "./static"
# Deprecated: use metricsServer.pprofEnable for authenticated profiling.
# Enable golang pprof handlers in admin listener.
webServer.pprofEnable = false
# Separate health and optional profiling listener. frpc does not serve /metrics.
# Port 0 disables it. The default address only accepts local connections.
metricsServer.addr = "127.0.0.1"
metricsServer.port = 0
# Credentials and TLS are independent of webServer; /healthz stays anonymous.
# metricsServer.user = "monitor"
# metricsServer.password = "change-me"
# metricsServer.tls.certFile = "metrics.crt"
# metricsServer.tls.keyFile = "metrics.key"
# Profiling requires explicit opt-in, uses the above credentials, and excludes cmdline.
metricsServer.pprofEnable = false
# The maximum amount of time a dial to server will wait for a connect to complete. Default value is 10 seconds.
# transport.dialServerTimeout = 10
+15 -1
View File
@@ -75,13 +75,27 @@ webServer.password = "admin"
# dashboard assets directory(only for debug mode)
# webServer.assetsDir = "./static"
# Deprecated: use metricsServer.pprofEnable for authenticated profiling.
# Enable golang pprof handlers in dashboard listener.
# Dashboard port must be set first
webServer.pprofEnable = false
# enablePrometheus will export prometheus metrics on webServer in /metrics api.
# Deprecated: enablePrometheus exports /metrics on webServer only.
# Use metricsServer.port instead; it enables business metrics automatically.
enablePrometheus = true
# Separate health, Prometheus metrics, and optional profiling listener.
# Port 0 disables it. The default address only accepts local connections.
metricsServer.addr = "127.0.0.1"
metricsServer.port = 0
# Credentials and TLS are independent of webServer; /healthz stays anonymous.
# metricsServer.user = "monitor"
# metricsServer.password = "change-me"
# metricsServer.tls.certFile = "metrics.crt"
# metricsServer.tls.keyFile = "metrics.key"
# Profiling requires explicit opt-in, uses the above credentials, and excludes cmdline.
metricsServer.pprofEnable = false
# console or real logFile path like ./frps.log
log.to = "./frps.log"
# trace, debug, info, warn, error
+2
View File
@@ -62,6 +62,7 @@ type ClientCommonConfig struct {
Log LogConfig `json:"log,omitempty"`
WebServer WebServerConfig `json:"webServer,omitempty"`
MetricsServer MetricsServerConfig `json:"metricsServer,omitempty"`
Transport ClientTransportConfig `json:"transport,omitempty"`
VirtualNet VirtualNetConfig `json:"virtualNet,omitempty"`
@@ -94,6 +95,7 @@ func (c *ClientCommonConfig) Complete() error {
c.Log.Complete()
c.Transport.Complete()
c.WebServer.Complete()
c.MetricsServer.Complete()
c.UDPPacketSize = util.EmptyOr(c.UDPPacketSize, 1500)
return nil
+20
View File
@@ -72,6 +72,26 @@ func (c *WebServerConfig) Complete() {
c.Addr = util.EmptyOr(c.Addr, "127.0.0.1")
}
// MetricsServerConfig configures a separate listener for health checks, metrics
// (frps only), and optional pprof handlers. It does not serve the web interface.
type MetricsServerConfig struct {
// Addr defaults to "127.0.0.1".
Addr string `json:"addr,omitempty"`
// Port disables the listener when set to 0.
Port int `json:"port,omitempty"`
// User and Password protect metrics and pprof; health checks remain anonymous.
User string `json:"user,omitempty"`
Password string `json:"password,omitempty"`
// PprofEnable explicitly enables profiling, excluding cmdline.
PprofEnable bool `json:"pprofEnable,omitempty"`
// Enable TLS if TLS is not nil.
TLS *TLSConfig `json:"tls,omitempty"`
}
func (c *MetricsServerConfig) Complete() {
c.Addr = util.EmptyOr(c.Addr, "127.0.0.1")
}
type TLSConfig struct {
// CertFile specifies the path of the cert file that client will load.
CertFile string `json:"certFile,omitempty"`
+2
View File
@@ -70,6 +70,7 @@ type ServerConfig struct {
SSHTunnelGateway SSHTunnelGateway `json:"sshTunnelGateway,omitempty"`
WebServer WebServerConfig `json:"webServer,omitempty"`
MetricsServer MetricsServerConfig `json:"metricsServer,omitempty"`
// EnablePrometheus will export prometheus metrics on webserver address
// in /metrics api.
EnablePrometheus bool `json:"enablePrometheus,omitempty"`
@@ -105,6 +106,7 @@ func (c *ServerConfig) Complete() error {
c.Log.Complete()
c.Transport.Complete()
c.WebServer.Complete()
c.MetricsServer.Complete()
c.SSHTunnelGateway.Complete()
c.BindAddr = util.EmptyOr(c.BindAddr, "0.0.0.0")
+1
View File
@@ -38,6 +38,7 @@ func (v *ConfigValidator) ValidateClientCommonConfig(c *v1.ClientCommonConfig) (
func() (Warning, error) { return v.validateAuthConfig(&c.Auth) },
func() (Warning, error) { return nil, validateLogConfig(&c.Log) },
func() (Warning, error) { return nil, validateWebServerConfig(&c.WebServer) },
func() (Warning, error) { return nil, validateMetricsServerConfig(&c.MetricsServer, &c.WebServer) },
func() (Warning, error) { return validateTransportConfig(&c.Transport) },
func() (Warning, error) { return validateIncludeFiles(c.IncludeConfigFiles) },
}
+21
View File
@@ -37,6 +37,27 @@ func validateWebServerConfig(c *v1.WebServerConfig) error {
return ValidatePort(c.Port, "webServer.port")
}
func validateMetricsServerConfig(c *v1.MetricsServerConfig, web *v1.WebServerConfig) error {
if err := ValidatePort(c.Port, "metricsServer.port"); err != nil {
return err
}
if c.TLS != nil {
if c.TLS.UseSystemCa {
return fmt.Errorf("metricsServer.tls.useSystemCa is only supported in frpc transport.tls")
}
if c.TLS.CertFile == "" || c.TLS.KeyFile == "" {
return fmt.Errorf("metricsServer.tls.certFile and metricsServer.tls.keyFile must both be specified when tls is enabled")
}
}
// Only reject an exact duplicate. Whether two different addresses can share
// a port depends on the address family and the platform, so leave it to
// net.Listen to decide.
if c.Port > 0 && web.Port > 0 && c.Port == web.Port && c.Addr == web.Addr {
return fmt.Errorf("metricsServer.addr:port conflicts with webServer.addr:port")
}
return nil
}
// ValidatePort checks that the network port is in range
func ValidatePort(port int, fieldPath string) error {
if 0 <= port && port <= 65535 {
@@ -0,0 +1,119 @@
// Copyright 2026 The frp Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package validation
import (
"testing"
"github.com/stretchr/testify/require"
v1 "github.com/fatedier/frp/pkg/config/v1"
)
func TestValidateMetricsServerConfig(t *testing.T) {
for _, tc := range []struct {
name string
metrics v1.MetricsServerConfig
web v1.WebServerConfig
wantErr string
}{
{name: "disabled"},
{name: "negative port", metrics: v1.MetricsServerConfig{Port: -1}, wantErr: "metricsServer.port"},
{name: "overflow port", metrics: v1.MetricsServerConfig{Port: 65536}, wantErr: "metricsServer.port"},
{name: "maximum port", metrics: v1.MetricsServerConfig{Port: 65535}},
{name: "empty TLS", metrics: v1.MetricsServerConfig{TLS: &v1.TLSConfig{}}, wantErr: "metricsServer.tls.certFile"},
{name: "only certificate", metrics: v1.MetricsServerConfig{TLS: &v1.TLSConfig{CertFile: "cert"}}, wantErr: "metricsServer.tls.keyFile"},
{name: "only key", metrics: v1.MetricsServerConfig{TLS: &v1.TLSConfig{KeyFile: "key"}}, wantErr: "metricsServer.tls.certFile"},
{name: "TLS pair", metrics: v1.MetricsServerConfig{TLS: &v1.TLSConfig{CertFile: "cert", KeyFile: "key"}}},
{name: "system CA", metrics: v1.MetricsServerConfig{TLS: &v1.TLSConfig{UseSystemCa: true}}, wantErr: "metricsServer.tls.useSystemCa"},
{
name: "same address", metrics: v1.MetricsServerConfig{Addr: "127.0.0.1", Port: 7501},
web: v1.WebServerConfig{Addr: "127.0.0.1", Port: 7501}, wantErr: "conflicts with webServer",
},
{
name: "different addresses", metrics: v1.MetricsServerConfig{Addr: "127.0.0.1", Port: 7501},
web: v1.WebServerConfig{Addr: "127.0.0.2", Port: 7501},
},
{
name: "different ports", metrics: v1.MetricsServerConfig{Port: 7501},
web: v1.WebServerConfig{Port: 7500},
},
{
name: "web disabled", metrics: v1.MetricsServerConfig{Port: 7501},
},
{
name: "metrics disabled", web: v1.WebServerConfig{Port: 7501},
},
} {
t.Run(tc.name, func(t *testing.T) {
checkMetricsConfigBoth(t, tc.metrics, tc.web, tc.wantErr)
})
}
// Two different addresses on the same port may or may not coexist depending
// on the address family and the platform, so only an exact duplicate is a
// configuration error here.
for _, wildcard := range []string{"", "0.0.0.0", "::"} {
t.Run("wildcard "+wildcard+" with loopback", func(t *testing.T) {
checkMetricsConfigBoth(t, v1.MetricsServerConfig{Addr: wildcard, Port: 7501},
v1.WebServerConfig{Addr: "127.0.0.1", Port: 7501}, "")
checkMetricsConfigBoth(t, v1.MetricsServerConfig{Addr: "127.0.0.1", Port: 7501},
v1.WebServerConfig{Addr: wildcard, Port: 7501}, "")
})
t.Run("same wildcard "+wildcard, func(t *testing.T) {
checkMetricsConfigBoth(t, v1.MetricsServerConfig{Addr: wildcard, Port: 7501},
v1.WebServerConfig{Addr: wildcard, Port: 7501}, "conflicts with webServer")
})
}
t.Run("different wildcards on the same port", func(t *testing.T) {
checkMetricsConfigBoth(t, v1.MetricsServerConfig{Addr: "0.0.0.0", Port: 7501},
v1.WebServerConfig{Addr: "::", Port: 7501}, "")
})
}
func checkMetricsConfigBoth(t *testing.T, metrics v1.MetricsServerConfig, web v1.WebServerConfig, wantErr string) {
t.Helper()
server := &v1.ServerConfig{}
client := &v1.ClientCommonConfig{}
require.NoError(t, server.Complete())
require.NoError(t, client.Complete())
// Assign after completion to also check raw empty (wildcard) addresses.
server.MetricsServer, server.WebServer = metrics, web
client.MetricsServer, client.WebServer = metrics, web
validator := NewConfigValidator(nil)
_, serverErr := validator.ValidateServerConfig(server)
_, clientErr := validator.ValidateClientCommonConfig(client)
for _, err := range []error{serverErr, clientErr} {
if wantErr == "" {
require.NoError(t, err)
} else {
require.ErrorContains(t, err, wantErr)
require.ErrorContains(t, err, "metricsServer")
}
}
}
func TestMetricsServerDefaults(t *testing.T) {
for _, port := range []int{0, 7501} {
server := &v1.ServerConfig{MetricsServer: v1.MetricsServerConfig{Port: port}}
client := &v1.ClientCommonConfig{MetricsServer: v1.MetricsServerConfig{Port: port}}
require.NoError(t, server.Complete())
require.NoError(t, client.Complete())
for _, cfg := range []v1.MetricsServerConfig{server.MetricsServer, client.MetricsServer} {
require.Equal(t, "127.0.0.1", cfg.Addr)
require.Equal(t, port, cfg.Port)
require.False(t, cfg.PprofEnable)
}
}
}
+3
View File
@@ -44,6 +44,9 @@ func (v *ConfigValidator) ValidateServerConfig(c *v1.ServerConfig) (Warning, err
if err := validateWebServerConfig(&c.WebServer); err != nil {
errs = AppendError(errs, err)
}
if err := validateMetricsServerConfig(&c.MetricsServer, &c.WebServer); err != nil {
errs = AppendError(errs, err)
}
if c.Transport.TLS.UseSystemCa {
errs = AppendError(errs, fmt.Errorf("transport.tls.useSystemCa is only supported in frpc transport.tls"))
}
+54
View File
@@ -0,0 +1,54 @@
// Copyright 2026 The frp Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package http
import (
"net/http"
"net/http/pprof"
v1 "github.com/fatedier/frp/pkg/config/v1"
)
// NewMetricsServer creates a listener with only health, optional metrics, and
// opt-in profiling endpoints. A zero port disables it and returns a nil server.
func NewMetricsServer(cfg v1.MetricsServerConfig, metricsHandler http.Handler) (*Server, error) {
if cfg.Port == 0 {
return nil, nil
}
s, err := newServer(cfg.Addr, cfg.Port, cfg.User, cfg.Password, cfg.TLS)
if err != nil {
return nil, err
}
s.router.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
})
authed := s.router.NewRoute().Subrouter()
authed.Use(s.authMiddleware)
if metricsHandler != nil {
authed.Handle("/metrics", metricsHandler)
}
if cfg.PprofEnable {
// Use exact routes rather than a prefix so cmdline and unknown endpoints
// cannot reach pprof.Index's fallback dispatch.
authed.HandleFunc("/debug/pprof/", pprof.Index)
authed.HandleFunc("/debug/pprof/profile", pprof.Profile)
authed.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
authed.HandleFunc("/debug/pprof/trace", pprof.Trace)
for _, name := range []string{"allocs", "block", "goroutine", "heap", "mutex", "threadcreate"} {
authed.Handle("/debug/pprof/"+name, pprof.Handler(name))
}
}
return s, nil
}
+111
View File
@@ -0,0 +1,111 @@
// Copyright 2026 The frp Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package http
import (
"net"
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/require"
"github.com/fatedier/frp/assets"
v1 "github.com/fatedier/frp/pkg/config/v1"
)
func newTestMetricsServer(t *testing.T, pprofEnabled bool, handler http.Handler) *Server {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
port := ln.Addr().(*net.TCPAddr).Port
require.NoError(t, ln.Close())
s, err := NewMetricsServer(v1.MetricsServerConfig{
Addr: "127.0.0.1", Port: port, User: "monitor", Password: "secret", PprofEnable: pprofEnabled,
}, handler)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, s.Close()) })
return s
}
func TestMetricsServerRoutes(t *testing.T) {
for _, enabled := range []bool{false, true} {
name := "pprof disabled"
if enabled {
name = "pprof enabled"
}
t.Run(name, func(t *testing.T) {
calls := 0
s := newTestMetricsServer(t, enabled, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
calls++
_, _ = w.Write([]byte("business metrics"))
}))
for _, authed := range []bool{false, true} {
for _, path := range []string{
"/healthz", "/metrics", "/debug/pprof/", "/debug/pprof/goroutine",
"/debug/pprof/cmdline", "/debug/pprof/unknown", "/", "/static/app.js", "/api/serverinfo", "/api/config",
} {
req := httptest.NewRequest(http.MethodGet, path, nil)
if authed {
req.SetBasicAuth("monitor", "secret")
}
resp := httptest.NewRecorder()
s.router.ServeHTTP(resp, req)
expected := http.StatusNotFound
switch path {
case "/healthz":
expected = http.StatusOK
require.Empty(t, resp.Body.String())
case "/metrics":
expected = http.StatusUnauthorized
if authed {
expected = http.StatusOK
require.Equal(t, "business metrics", resp.Body.String())
}
case "/debug/pprof/", "/debug/pprof/goroutine":
if enabled {
expected = http.StatusUnauthorized
if authed {
expected = http.StatusOK
}
}
}
require.Equal(t, expected, resp.Code, "path=%s authenticated=%v", path, authed)
}
}
require.Equal(t, 1, calls, "unauthenticated requests must not reach the metrics handler")
})
}
}
func TestMetricsServerWithoutMetricsPreservesAssets(t *testing.T) {
assets.Load(t.TempDir())
before := assets.FileSystem
t.Cleanup(func() { assets.Load("") })
s := newTestMetricsServer(t, false, nil)
// assets.Load uses http.Dir for local assets, a value rather than a pointer.
require.Equal(t, before, assets.FileSystem)
req := httptest.NewRequest(http.MethodGet, "/metrics", nil)
req.SetBasicAuth("monitor", "secret")
resp := httptest.NewRecorder()
s.router.ServeHTTP(resp, req)
require.Equal(t, http.StatusNotFound, resp.Code)
}
func TestMetricsServerDisabled(t *testing.T) {
s, err := NewMetricsServer(v1.MetricsServerConfig{TLS: &v1.TLSConfig{CertFile: "nonexistent"}}, nil)
require.NoError(t, err)
require.Nil(t, s)
}
+15 -7
View File
@@ -48,7 +48,18 @@ type Server struct {
func NewServer(cfg v1.WebServerConfig) (*Server, error) {
assets.Load(cfg.AssetsDir)
addr := net.JoinHostPort(cfg.Addr, strconv.Itoa(cfg.Port))
s, err := newServer(cfg.Addr, cfg.Port, cfg.User, cfg.Password, cfg.TLS)
if err != nil {
return nil, err
}
if cfg.PprofEnable {
s.registerPprofHandlers()
}
return s, nil
}
func newServer(bindAddr string, port int, user, password string, tlsConfig *v1.TLSConfig) (*Server, error) {
addr := net.JoinHostPort(bindAddr, strconv.Itoa(port))
if addr == ":" {
addr = ":http"
}
@@ -65,11 +76,8 @@ func NewServer(cfg v1.WebServerConfig) (*Server, error) {
hs: hs,
router: router,
}
if cfg.PprofEnable {
s.registerPprofHandlers()
}
if cfg.TLS != nil {
cert, err := tls.LoadX509KeyPair(cfg.TLS.CertFile, cfg.TLS.KeyFile)
if tlsConfig != nil {
cert, err := tls.LoadX509KeyPair(tlsConfig.CertFile, tlsConfig.KeyFile)
if err != nil {
return nil, err
}
@@ -77,7 +85,7 @@ func NewServer(cfg v1.WebServerConfig) (*Server, error) {
Certificates: []tls.Certificate{cert},
}
}
s.authMiddleware = netpkg.NewHTTPAuthMiddleware(cfg.User, cfg.Password).SetAuthFailDelay(200 * time.Millisecond).Middleware
s.authMiddleware = netpkg.NewHTTPAuthMiddleware(user, password).SetAuthFailDelay(200 * time.Millisecond).Middleware
// Bind only after all fallible preparation so an error cannot leak the listener.
ln, err := net.Listen("tcp", addr)
+108
View File
@@ -0,0 +1,108 @@
// Copyright 2026 The frp Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package server
import (
"context"
"fmt"
"io"
"net"
"net/http"
"path/filepath"
"testing"
"time"
"github.com/stretchr/testify/require"
v1 "github.com/fatedier/frp/pkg/config/v1"
)
func TestServiceMetricsServerWithoutWebServer(t *testing.T) {
cfg := &v1.ServerConfig{
BindAddr: "127.0.0.1", ProxyBindAddr: "127.0.0.1",
MetricsServer: v1.MetricsServerConfig{Port: freeLifecycleTCPPort(t)},
}
require.NoError(t, cfg.Complete())
cfg.BindPort = 0
svr, err := NewService(cfg)
require.NoError(t, err)
require.Nil(t, svr.webServer)
address := svr.metricsServer.Address()
done := make(chan struct{})
go func() { svr.Run(context.Background()); close(done) }()
defer func() {
require.NoError(t, svr.Close())
waitForSignal(t, done, "metrics service shutdown")
assertLifecyclePorts(t, []string{address}, true)
}()
client := &http.Client{Timeout: 3 * time.Second}
resp, err := client.Get("http://" + address + "/healthz")
require.NoError(t, err)
defer resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.Empty(t, body)
}
func TestNewServiceReleasesResourcesOnMetricsServerFailure(t *testing.T) {
for _, mode := range []string{"port conflict", "invalid TLS certificate"} {
t.Run(mode, func(t *testing.T) {
cfg := &v1.ServerConfig{BindAddr: "127.0.0.1", ProxyBindAddr: "127.0.0.1"}
require.NoError(t, cfg.Complete())
// Reserve distinct ports together, including the second HTTP listener.
ports := []*int{&cfg.BindPort, &cfg.VhostHTTPPort, &cfg.WebServer.Port, &cfg.MetricsServer.Port}
reservations := make([]net.Listener, 0, len(ports))
addresses := make([]string, 0, len(ports))
for _, port := range ports {
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
*port = ln.Addr().(*net.TCPAddr).Port
reservations = append(reservations, ln)
addresses = append(addresses, ln.Addr().String())
}
for _, ln := range reservations[:len(reservations)-1] {
require.NoError(t, ln.Close())
}
if mode == "invalid TLS certificate" {
require.NoError(t, reservations[len(reservations)-1].Close())
cfg.MetricsServer.TLS = &v1.TLSConfig{
CertFile: filepath.Join(t.TempDir(), "missing.crt"), KeyFile: "missing.key",
}
} else {
// This port belongs to the test, not to the failed service.
addresses = addresses[:len(addresses)-1]
}
workersBefore := countPortManagerWorkers(t)
for range 5 {
svr, err := NewService(cfg)
if svr != nil {
_ = svr.Close()
}
require.Error(t, err)
require.Nil(t, svr)
if mode == "port conflict" {
require.ErrorContains(t, err, fmt.Sprint(cfg.MetricsServer.Port))
} else {
require.ErrorContains(t, err, "missing.crt")
}
assertLifecyclePorts(t, addresses, true)
}
require.Equal(t, workersBefore, countPortManagerWorkers(t), "failed construction leaked port-manager workers")
})
}
}
+52 -6
View File
@@ -31,6 +31,7 @@ import (
"github.com/fatedier/golib/crypto"
"github.com/fatedier/golib/net/mux"
fmux "github.com/fatedier/yamux"
"github.com/prometheus/client_golang/prometheus/promhttp"
quic "github.com/quic-go/quic-go"
"github.com/samber/lo"
@@ -121,6 +122,7 @@ type Service struct {
// web server for dashboard UI and apis
webServer *httppkg.Server
metricsServer *httppkg.Server
// HTTP vhost resources are started and owned by NewService.
httpVhostServer *http.Server
httpVhostListener net.Listener
@@ -347,8 +349,8 @@ func NewService(cfg *v1.ServerConfig) (*Service, error) {
}
svr.rc.NatHoleController = nc
// Create the web server after all fallible steps so its listener is not
// leaked when an earlier error causes NewService to return.
// Create HTTP servers after all other fallible steps, rolling back listeners
// if either server cannot be created.
if cfg.WebServer.Port > 0 {
ws, err := httppkg.NewServer(cfg.WebServer)
if err != nil {
@@ -357,11 +359,24 @@ func NewService(cfg *v1.ServerConfig) (*Service, error) {
return nil, err
}
svr.webServer = ws
modelmetrics.EnableMem()
if cfg.EnablePrometheus {
modelmetrics.EnablePrometheus()
}
if cfg.MetricsServer.Port > 0 {
ms, err := httppkg.NewMetricsServer(cfg.MetricsServer, promhttp.Handler())
if err != nil {
_ = svr.Close()
return nil, err
}
svr.metricsServer = ms
}
// Collectors are enabled at most once. The in-memory collector is only read
// by the dashboard API, and Prometheus metrics need an endpoint to serve them.
enableMem, enablePrometheus := metricsCollectors(cfg)
if enableMem {
modelmetrics.EnableMem()
}
if enablePrometheus {
modelmetrics.EnablePrometheus()
}
// Start port-manager cleanup workers only after all fallible steps have succeeded.
@@ -375,6 +390,16 @@ func NewService(cfg *v1.ServerConfig) (*Service, error) {
return svr, nil
}
// metricsCollectors reports which collectors should be started for cfg. The
// in-memory collector only backs the dashboard API (webServer), while the
// Prometheus collector needs a listener that serves /metrics: either the new
// metricsServer, or the legacy webServer endpoint gated by enablePrometheus.
func metricsCollectors(cfg *v1.ServerConfig) (mem bool, prometheus bool) {
mem = cfg.WebServer.Port > 0
prometheus = cfg.MetricsServer.Port > 0 || (cfg.EnablePrometheus && mem)
return mem, prometheus
}
func (svr *Service) Run(ctx context.Context) {
ctx, cancel := context.WithCancel(ctx)
svr.lifecycleMu.Lock()
@@ -396,6 +421,15 @@ func (svr *Service) Run(ctx context.Context) {
// run dashboard web server.
if svr.webServer != nil {
if svr.cfg.WebServer.PprofEnable {
log.Warnf("webServer pprof endpoints are deprecated; use metricsServer.pprofEnable instead")
}
if svr.cfg.EnablePrometheus {
log.Warnf("webServer /metrics is deprecated; configure metricsServer.port instead")
}
if svr.cfg.MetricsServer.Port == 0 {
log.Infof("webServer /healthz is deprecated; health probes can migrate to metricsServer.port")
}
go func() {
log.Infof("dashboard listen on %s", svr.webServer.Address())
if err := svr.webServer.Run(); err != nil {
@@ -404,6 +438,15 @@ func (svr *Service) Run(ctx context.Context) {
}()
}
if svr.metricsServer != nil {
go func() {
log.Infof("metrics server listen on %s", svr.metricsServer.Address())
if err := svr.metricsServer.Run(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Warnf("metrics server exit with error: %v", err)
}
}()
}
go svr.HandleListener(svr.sshTunnelListener, true)
if svr.kcpListener != nil {
@@ -462,6 +505,9 @@ func (svr *Service) closeResources() {
if svr.webServer != nil {
svr.webServer.Close()
}
if svr.metricsServer != nil {
svr.metricsServer.Close()
}
if svr.httpVhostServer != nil {
svr.httpVhostServer.Close()
// Close even if Serve has not registered its listener yet.
+44
View File
@@ -276,6 +276,50 @@ func TestServiceRunAfterEarlyShutdown(t *testing.T) {
}
}
func TestMetricsCollectors(t *testing.T) {
for _, tc := range []struct {
name string
webPort, metricsPort int
enablePrometheus bool
wantMem, wantPrometheus bool
}{
{name: "nothing enabled"},
{name: "web server only", webPort: 7500, wantMem: true},
{
name: "web server with enablePrometheus", webPort: 7500, enablePrometheus: true,
wantMem: true, wantPrometheus: true,
},
{
// enablePrometheus alone has no endpoint to serve metrics from, so the
// Prometheus collector must stay off, as it did before metricsServer.
name: "enablePrometheus without listeners", enablePrometheus: true,
},
{
// The in-memory collector only backs the dashboard API.
name: "metrics server only", metricsPort: 7501, wantPrometheus: true,
},
{
name: "both servers", webPort: 7500, metricsPort: 7501,
wantMem: true, wantPrometheus: true,
},
{
name: "both servers with enablePrometheus", webPort: 7500, metricsPort: 7501, enablePrometheus: true,
wantMem: true, wantPrometheus: true,
},
} {
t.Run(tc.name, func(t *testing.T) {
cfg := &v1.ServerConfig{
WebServer: v1.WebServerConfig{Port: tc.webPort},
MetricsServer: v1.MetricsServerConfig{Port: tc.metricsPort},
EnablePrometheus: tc.enablePrometheus,
}
mem, prometheus := metricsCollectors(cfg)
require.Equal(t, tc.wantMem, mem, "EnableMem")
require.Equal(t, tc.wantPrometheus, prometheus, "EnablePrometheus")
})
}
}
func TestServiceCloseMarksClosedBeforeCleanup(t *testing.T) {
svr, listener, addresses := newLifecycleTestService(t, false)
listener.closeEntered, listener.closeResume = make(chan struct{}), make(chan struct{})
+124
View File
@@ -0,0 +1,124 @@
package features
import (
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
"github.com/onsi/ginkgo/v2"
"github.com/onsi/gomega"
"github.com/fatedier/frp/server/http/model"
"github.com/fatedier/frp/test/e2e/framework"
"github.com/fatedier/frp/test/e2e/framework/consts"
"github.com/fatedier/frp/test/e2e/pkg/request"
)
func metricsServerConfig(port int, pprof bool) string {
return fmt.Sprintf(`
metricsServer.port = %d
metricsServer.user = "monitor"
metricsServer.password = "secret"
metricsServer.pprofEnable = %t
`, port, pprof)
}
func metricsRequest(port int, path string, authenticated bool) *request.Request {
r := framework.NewRequest().HTTP().Port(port).HTTPPath(path)
if authenticated {
r.HTTPAuth("monitor", "secret")
}
return r
}
var _ = ginkgo.Describe("[Feature: Metrics Server]", func() {
f := framework.NewDefaultFramework()
// A single connected client must produce exactly one event, including when
// both listeners expose /metrics. Checking names alone would also pass with
// registered collectors that never receive business events.
expectBusinessMetrics := func(port int) {
gomega.Eventually(func() bool {
resp, err := metricsRequest(port, "/metrics", true).Do()
return err == nil && resp.Code == http.StatusOK &&
strings.Contains(string(resp.Content), "\nfrp_server_client_counts 1\n")
}, 5*time.Second, 50*time.Millisecond).Should(gomega.BeTrue())
}
ginkgo.It("serves anonymous health alongside both web servers", func() {
serverMetrics, clientMetrics := f.AllocPort(), f.AllocPort()
serverWeb, clientWeb := f.AllocPort(), f.AllocPort()
webConfig := func(port int) string {
return fmt.Sprintf(`
webServer.port = %d
webServer.user = "admin"
webServer.password = "admin-secret"
`, port)
}
serverConf := consts.DefaultServerConfig + metricsServerConfig(serverMetrics, false) + webConfig(serverWeb)
clientConf := consts.DefaultClientConfig + metricsServerConfig(clientMetrics, false) + webConfig(clientWeb)
f.RunProcesses(serverConf, []string{clientConf})
for _, port := range []int{serverMetrics, clientMetrics, serverWeb, clientWeb} {
framework.NewRequestExpect(f).Request(metricsRequest(port, "/healthz", false)).ExpectResp([]byte("")).
Ensure(framework.ExpectResponseCode(http.StatusOK), func(resp *request.Response) bool { return len(resp.Content) == 0 })
}
for _, port := range []int{serverMetrics, clientMetrics} {
for _, path := range []string{"/", "/api/serverinfo", "/api/config", "/static/app.js"} {
framework.NewRequestExpect(f).Request(metricsRequest(port, path, true)).Ensure(framework.ExpectResponseCode(http.StatusNotFound))
}
}
})
ginkgo.It("authenticates business metrics without enablePrometheus", func() {
port := f.AllocPort()
serverConf := consts.DefaultServerConfig + metricsServerConfig(port, false) + fmt.Sprintf("\nwebServer.port = %d\n", f.AllocPort())
f.RunProcesses(serverConf, []string{consts.DefaultClientConfig})
framework.NewRequestExpect(f).Request(metricsRequest(port, "/metrics", false)).Ensure(framework.ExpectResponseCode(http.StatusUnauthorized))
expectBusinessMetrics(port)
})
ginkgo.It("requires opt-in and authentication for pprof and excludes cmdline", func() {
serverPort, clientPort := f.AllocPort(), f.AllocPort()
f.RunProcesses(consts.DefaultServerConfig+metricsServerConfig(serverPort, true),
[]string{consts.DefaultClientConfig + metricsServerConfig(clientPort, false)})
for _, path := range []string{"/debug/pprof/", "/debug/pprof/goroutine"} {
framework.NewRequestExpect(f).Request(metricsRequest(serverPort, path, false)).Ensure(framework.ExpectResponseCode(http.StatusUnauthorized))
framework.NewRequestExpect(f).Request(metricsRequest(serverPort, path, true)).Ensure(framework.ExpectResponseCode(http.StatusOK))
framework.NewRequestExpect(f).Request(metricsRequest(clientPort, path, true)).Ensure(framework.ExpectResponseCode(http.StatusNotFound))
}
for _, authenticated := range []bool{false, true} {
framework.NewRequestExpect(f).Request(metricsRequest(serverPort, "/debug/pprof/cmdline", authenticated)).
Ensure(framework.ExpectResponseCode(http.StatusNotFound))
}
})
ginkgo.It("serves health and business metrics without either web server", func() {
serverPort, clientPort := f.AllocPort(), f.AllocPort()
f.RunProcesses(consts.DefaultServerConfig+metricsServerConfig(serverPort, false),
[]string{consts.DefaultClientConfig + metricsServerConfig(clientPort, false)})
for _, port := range []int{serverPort, clientPort} {
framework.NewRequestExpect(f).Request(metricsRequest(port, "/healthz", false)).Ensure(framework.ExpectResponseCode(http.StatusOK))
}
expectBusinessMetrics(serverPort)
framework.NewRequestExpect(f).Request(metricsRequest(clientPort, "/metrics", true)).Ensure(framework.ExpectResponseCode(http.StatusNotFound))
})
ginkgo.It("collects once when both metrics endpoints are enabled", func() {
metricsPort, webPort := f.AllocPort(), f.AllocPort()
serverConf := consts.DefaultServerConfig + metricsServerConfig(metricsPort, false) + fmt.Sprintf(`
enablePrometheus = true
webServer.port = %d
webServer.user = "monitor"
webServer.password = "secret"
`, webPort)
f.RunProcesses(serverConf, []string{consts.DefaultClientConfig})
expectBusinessMetrics(metricsPort)
expectBusinessMetrics(webPort)
framework.NewRequestExpect(f).Request(metricsRequest(webPort, "/api/serverinfo", true)).Ensure(func(resp *request.Response) bool {
var info model.ServerInfoResp
return resp.Code == http.StatusOK && json.Unmarshal(resp.Content, &info) == nil && info.ClientCounts == 1
})
})
})