Compare commits

2 Commits
Author SHA1 Message Date
Mxmilu666 c879e10ac1 feat(tls): add support for RDP negotiation before TLS handshake 2026-09-30 16:54:53 +08:00
Mxmilu666andClaude Opus 5.5 ae7135ab64 feat(tls): reassemble hello messages split across TLS records
A handshake message may span any number of records, and splitting the
ClientHello into tiny records hides the SNI from analyzers that expect it
in the first one. The tls analyzer read the first record header and then
cut the handshake body straight out of the stream, so fragmented hellos
either stalled until Limit() or failed to parse, and every SNI rule was
bypassed.

The record layer is now peeled off and the hello rebuilt from the record
bodies before parsing, for both ClientHello and ServerHello. The number of
records a hello spanned is exposed as tls.req_records / tls.resp_records,
kept outside req/resp so a non-nil req still means a fully parsed hello.
Fragmentation is published as soon as a second record shows up, since
1-byte records can stretch a hello past Limit() and it never completes.

docs: add TLS record fragmentation doc, correct the claim in the TCP
window evasion doc that tls is unaffected by splitting.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 17:31:57 +08:00
8 changed files with 753 additions and 201 deletions
+2
View File
@@ -39,6 +39,8 @@ Telegram 群组: https://t.me/OpGFW
- [Minecraft 分析器](docs/minecraft.zh.md)
- [EasyTier 分析器](docs/easytier.zh.md)
- [TCP 窗口操纵规避与检测](docs/tcp-window-evasion.zh.md)
- [TLS record 分片规避与检测](docs/tls-record-fragmentation.zh.md)
- [拦截 TLS、放行 RDP](docs/rdp-tls.zh.md)
## 使用场景
+183 -169
View File
@@ -22,38 +22,72 @@ func (a *TLSAnalyzer) NewTCP(info analyzer.TCPInfo, logger analyzer.Logger) anal
return newTLSStream(logger)
}
const (
// tlsRecordHeaderSize is content type (1 byte) + legacy protocol version
// (2 bytes) + length (2 bytes).
tlsRecordHeaderSize = 5
// tlsMaxPlaintextLen is the largest body a plaintext record may carry.
tlsMaxPlaintextLen = 1 << 14
// tlsHandshakeHeaderSize is message type (1 byte) + length (3 bytes).
tlsHandshakeHeaderSize = 4
)
type tlsStream struct {
logger analyzer.Logger
reqBuf *utils.ByteBuffer
reqMap analyzer.PropMap
reqUpdated bool
reqLSM *utils.LinearStateMachine
reqDone bool
req tlsHello
resp tlsHello
}
respBuf *utils.ByteBuffer
respMap analyzer.PropMap
respUpdated bool
respLSM *utils.LinearStateMachine
respDone bool
// tlsHello reads one hello message off one direction of the stream.
//
// A handshake message may be split across any number of records (RFC 8446
// 5.1), and splitting the ClientHello into tiny records is a common way to
// hide the SNI from middleboxes that expect it in the first record. So the
// record layer is peeled off first and the message is rebuilt from the record
// bodies before it is parsed.
type tlsHello struct {
rec utils.ByteBuffer // record layer bytes not consumed yet
msg utils.ByteBuffer // handshake layer bytes collected so far
lsm *utils.LinearStateMachine
clientHelloLen int
serverHelloLen int
msgType byte
x224Code byte // X.224 connection PDU that may precede the hello on RDP
minLen int // smallest valid message body
parse func(*utils.ByteBuffer) analyzer.PropMap // parses the message body
recLeft int // body bytes of the current record not consumed yet
msgLen int // message body length, -1 until the handshake header is read
records int // records that have carried the message so far
reported int // value of records last published
m analyzer.PropMap
updated bool
done bool
rdp analyzer.PropMap // RDP negotiation, nil if there was none
rdpUpdated bool
}
func newTLSStream(logger analyzer.Logger) *tlsStream {
s := &tlsStream{logger: logger, reqBuf: &utils.ByteBuffer{}, respBuf: &utils.ByteBuffer{}}
s.reqLSM = utils.NewLinearStateMachine(
s.tlsClientHelloPreprocess,
s.parseClientHelloData,
)
s.respLSM = utils.NewLinearStateMachine(
s.tlsServerHelloPreprocess,
s.parseServerHelloData,
)
s := &tlsStream{logger: logger}
// minimal ClientHello body: protocol version (2 bytes) + random (32 bytes) +
// + session ID (1 byte) + cipher suites (4 bytes) +
// + compression methods (2 bytes) + no extensions
s.req.init(internal.TypeClientHello, x224CodeConnectionRequest, 41, internal.ParseTLSClientHelloMsgData)
// minimal ServerHello body: server version (2 bytes) + random (32 bytes) +
// + session ID (>=1 byte) + cipher suite (2 bytes) +
// + compression method (1 byte) + no extensions
s.resp.init(internal.TypeServerHello, x224CodeConnectionConfirm, 38, internal.ParseTLSServerHelloMsgData)
return s
}
func (h *tlsHello) init(msgType, x224Code byte, minLen int, parse func(*utils.ByteBuffer) analyzer.PropMap) {
h.msgType, h.x224Code, h.minLen, h.parse = msgType, x224Code, minLen, parse
h.msgLen = -1
h.lsm = utils.NewLinearStateMachine(h.readPrelude, h.readMessage, h.parseMessage)
}
func (s *tlsStream) Feed(rev, start, end bool, skip int, data []byte) (u *analyzer.PropUpdate, done bool) {
if skip != 0 {
return nil, true
@@ -61,166 +95,146 @@ func (s *tlsStream) Feed(rev, start, end bool, skip int, data []byte) (u *analyz
if len(data) == 0 {
return nil, false
}
var update *analyzer.PropUpdate
var cancelled bool
h, key := &s.req, "req"
if rev {
s.respBuf.Append(data)
s.respUpdated = false
cancelled, s.respDone = s.respLSM.Run()
if s.respUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateMerge,
M: analyzer.PropMap{"resp": s.respMap},
}
s.respUpdated = false
h, key = &s.resp, "resp"
}
if h.done {
return nil, false
}
h.rec.Append(data)
h.updated = false
var cancelled bool
cancelled, h.done = h.lsm.Run()
// A fragmented hello is published as soon as it shows up, not only once
// the message is complete: a peer can make it too large to ever complete
// within Limit(), and a rule should not need the SNI to act on it.
if m := h.update(key, h.updated || (h.records > 1 && h.reported <= 1)); m != nil {
u = &analyzer.PropUpdate{Type: analyzer.PropUpdateMerge, M: m}
}
return u, cancelled || (s.req.done && s.resp.done)
}
// update returns the properties to publish for this direction, or nil if
// there is nothing to publish. The parsed message goes under key, the record
// count under key + "_records". The record count is kept out of the message
// map so that a non-nil message map still means a fully parsed hello. An RDP
// negotiation goes under "rdp_" + key as soon as it is parsed, whatever
// publish says.
func (h *tlsHello) update(key string, publish bool) analyzer.PropMap {
m := analyzer.PropMap{}
if h.rdpUpdated {
m["rdp_"+key] = h.rdp
h.rdpUpdated = false
}
if publish && h.records > 0 {
m[key+"_records"] = h.records
if h.updated {
m[key] = h.m
h.updated = false
}
} else {
s.reqBuf.Append(data)
s.reqUpdated = false
cancelled, s.reqDone = s.reqLSM.Run()
if s.reqUpdated {
update = &analyzer.PropUpdate{
Type: analyzer.PropUpdateMerge,
M: analyzer.PropMap{"req": s.reqMap},
h.reported = h.records
}
if len(m) == 0 {
return nil
}
return m
}
// readMessage consumes records until a whole hello message is collected in
// h.msg.
func (h *tlsHello) readMessage() utils.LSMAction {
for {
if h.msgLen < 0 && h.msg.Len() >= tlsHandshakeHeaderSize {
header := h.msg.Buf
if header[0] != h.msgType {
return utils.LSMActionCancel
}
s.reqUpdated = false
h.msgLen = int(header[1])<<16 | int(header[2])<<8 | int(header[3])
if h.msgLen < h.minLen {
return utils.LSMActionCancel
}
}
if h.msgLen >= 0 && h.msg.Len() >= tlsHandshakeHeaderSize+h.msgLen {
return utils.LSMActionNext
}
if h.recLeft == 0 {
header, ok := h.rec.Get(tlsRecordHeaderSize, true)
if !ok {
return utils.LSMActionPause
}
if header[0] != internal.RecordTypeHandshake {
// Nothing else may come between the records of a handshake
// message.
return utils.LSMActionCancel
}
h.recLeft = int(header[3])<<8 | int(header[4])
if h.recLeft > tlsMaxPlaintextLen {
return utils.LSMActionCancel
}
// Empty records are forbidden for handshake messages, but still
// counted rather than rejected: a peer that sends them anyway is
// fragmenting, and rejecting them would stop the analysis.
h.records++
continue
}
n := min(h.recLeft, h.rec.Len())
if n == 0 {
return utils.LSMActionPause
}
body, _ := h.rec.Get(n, true)
h.recLeft -= n
if h.msg.Len() == 0 {
// Usually the whole message sits in this one chunk, so reference
// it instead of copying. The capacity is capped so that a later
// Append reallocates rather than writing into h.rec.
h.msg.Buf = body[:n:n]
} else {
h.msg.Append(body)
}
}
return update, cancelled || (s.reqDone && s.respDone)
}
// tlsClientHelloPreprocess validates ClientHello message.
// parseMessage converts the collected hello message into h.m.
//
// During validation, message header and first handshake header may be removed
// from `s.reqBuf`.
func (s *tlsStream) tlsClientHelloPreprocess() utils.LSMAction {
// headers size: content type (1 byte) + legacy protocol version (2 bytes) +
// + content length (2 bytes) + message type (1 byte) +
// + handshake length (3 bytes)
const headersSize = 9
// minimal data size: protocol version (2 bytes) + random (32 bytes) +
// + session ID (1 byte) + cipher suites (4 bytes) +
// + compression methods (2 bytes) + no extensions
const minDataSize = 41
header, ok := s.reqBuf.Get(headersSize, true)
if !ok {
// not a full header yet
return utils.LSMActionPause
}
if header[0] != internal.RecordTypeHandshake || header[5] != internal.TypeClientHello {
return utils.LSMActionCancel
}
s.clientHelloLen = int(header[6])<<16 | int(header[7])<<8 | int(header[8])
if s.clientHelloLen < minDataSize {
return utils.LSMActionCancel
}
// TODO: something is missing. See:
// const messageHeaderSize = 4
// fullMessageLen := int(header[3])<<8 | int(header[4])
// msgNo := fullMessageLen / int(messageHeaderSize+s.serverHelloLen)
// if msgNo != 1 {
// // what here?
// }
// if messageNo != int(messageNo) {
// // what here?
// }
return utils.LSMActionNext
}
// tlsServerHelloPreprocess validates ServerHello message.
//
// During validation, message header and first handshake header may be removed
// from `s.reqBuf`.
func (s *tlsStream) tlsServerHelloPreprocess() utils.LSMAction {
// header size: content type (1 byte) + legacy protocol version (2 byte) +
// + content length (2 byte) + message type (1 byte) +
// + handshake length (3 byte)
const headersSize = 9
// minimal data size: server version (2 byte) + random (32 byte) +
// + session ID (>=1 byte) + cipher suite (2 byte) +
// + compression method (1 byte) + no extensions
const minDataSize = 38
header, ok := s.respBuf.Get(headersSize, true)
if !ok {
// not a full header yet
return utils.LSMActionPause
}
if header[0] != internal.RecordTypeHandshake || header[5] != internal.TypeServerHello {
return utils.LSMActionCancel
}
s.serverHelloLen = int(header[6])<<16 | int(header[7])<<8 | int(header[8])
if s.serverHelloLen < minDataSize {
return utils.LSMActionCancel
}
// TODO: something is missing. See example:
// const messageHeaderSize = 4
// fullMessageLen := int(header[3])<<8 | int(header[4])
// msgNo := fullMessageLen / int(messageHeaderSize+s.serverHelloLen)
// if msgNo != 1 {
// // what here?
// }
// if messageNo != int(messageNo) {
// // what here?
// }
return utils.LSMActionNext
}
// parseClientHelloData converts valid ClientHello message data (without
// headers) into `analyzer.PropMap`.
//
// Parsing error may leave `s.reqBuf` in an unusable state.
func (s *tlsStream) parseClientHelloData() utils.LSMAction {
chBuf, ok := s.reqBuf.GetSubBuffer(s.clientHelloLen, true)
if !ok {
// Not a full client hello yet
return utils.LSMActionPause
}
m := internal.ParseTLSClientHelloMsgData(chBuf)
// Anything after the message, such as the rest of the server's flight, is
// ignored.
func (h *tlsHello) parseMessage() utils.LSMAction {
body := &utils.ByteBuffer{Buf: h.msg.Buf[tlsHandshakeHeaderSize : tlsHandshakeHeaderSize+h.msgLen]}
m := h.parse(body)
h.rec.Reset()
h.msg.Reset()
if m == nil {
return utils.LSMActionCancel
} else {
s.reqUpdated = true
s.reqMap = m
return utils.LSMActionNext
}
}
// parseServerHelloData converts valid ServerHello message data (without
// headers) into `analyzer.PropMap`.
//
// Parsing error may leave `s.respBuf` in an unusable state.
func (s *tlsStream) parseServerHelloData() utils.LSMAction {
shBuf, ok := s.respBuf.GetSubBuffer(s.serverHelloLen, true)
if !ok {
// Not a full server hello yet
return utils.LSMActionPause
}
m := internal.ParseTLSServerHelloMsgData(shBuf)
if m == nil {
return utils.LSMActionCancel
} else {
s.respUpdated = true
s.respMap = m
return utils.LSMActionNext
}
h.m = m
h.updated = true
return utils.LSMActionNext
}
func (s *tlsStream) Close(limited bool) *analyzer.PropUpdate {
s.reqBuf.Reset()
s.respBuf.Reset()
s.reqMap = nil
s.respMap = nil
return nil
// Report how far a fragmented hello got if it never completed, e.g.
// because it was stretched past Limit().
m := s.req.update("req", s.req.records > 1 && s.req.records > s.req.reported)
if rm := s.resp.update("resp", s.resp.records > 1 && s.resp.records > s.resp.reported); rm != nil {
if m == nil {
m = rm
} else {
for k, v := range rm {
m[k] = v
}
}
}
s.req.close()
s.resp.close()
if m == nil {
return nil
}
return &analyzer.PropUpdate{Type: analyzer.PropUpdateMerge, M: m}
}
func (h *tlsHello) close() {
h.rec.Reset()
h.msg.Reset()
h.m = nil
}
+123
View File
@@ -0,0 +1,123 @@
package tcp
import (
"bytes"
"encoding/binary"
"github.com/apernet/OpenGFW/analyzer"
"github.com/apernet/OpenGFW/analyzer/internal"
"github.com/apernet/OpenGFW/analyzer/utils"
)
// RDP does not start TLS right away: the client first sends an X.224
// Connection Request and the server answers with a Connection Confirm, each in
// a TPKT packet, to agree on a security protocol ([MS-RDPBCGR] 1.3.1.1). Only
// then does the TLS handshake begin, on the same connection.
//
// The tls analyzer used to give up on the first byte of such a stream, which
// left RDP invisible to TLS rules, and with it anything that puts a fake RDP
// negotiation in front of its TLS. So the negotiation is parsed as an optional
// prelude of the hello, and published as rdp_req / rdp_resp.
const (
tpktVersion = 0x03
tpktHeaderSize = 4 // version (1 byte) + reserved (1 byte) + length (2 bytes)
// x224FixedSize is length indicator (1 byte) + code (1 byte) +
// DST-REF (2 bytes) + SRC-REF (2 bytes) + class option (1 byte).
x224FixedSize = 7
x224CodeConnectionRequest = 0xe0
x224CodeConnectionConfirm = 0xd0
rdpNegReq = 0x01
rdpNegRsp = 0x02
rdpNegFailure = 0x03
rdpNegSize = 8
)
var rdpCookiePrefix = []byte("Cookie: ")
// readPrelude consumes the X.224 connection PDU that precedes the hello on an
// RDP connection, if there is one.
func (h *tlsHello) readPrelude() utils.LSMAction {
b, ok := h.rec.GetByte(false)
if !ok {
return utils.LSMActionPause
}
if b == internal.RecordTypeHandshake {
return utils.LSMActionNext
}
if b != tpktVersion {
return utils.LSMActionCancel
}
header, ok := h.rec.Get(tpktHeaderSize, false)
if !ok {
return utils.LSMActionPause
}
n := int(binary.BigEndian.Uint16(header[2:4]))
if header[1] != 0 || n < tpktHeaderSize+x224FixedSize {
return utils.LSMActionCancel
}
pdu, ok := h.rec.Get(n, true)
if !ok {
return utils.LSMActionPause
}
m := parseX224Connection(pdu[tpktHeaderSize:], h.x224Code)
if m == nil {
return utils.LSMActionCancel
}
h.rdp = m
h.rdpUpdated = true
return utils.LSMActionNext
}
// parseX224Connection parses an X.224 Connection Request or Confirm TPDU
// carrying the RDP negotiation, or returns nil if tpdu is not one.
func parseX224Connection(tpdu []byte, code byte) analyzer.PropMap {
// The length indicator does not count itself. The low nibble of the code
// is the credit, and the high nibble of the class option the class, which
// is always 0 for RDP.
if int(tpdu[0]) != len(tpdu)-1 || tpdu[1]&0xf0 != code || tpdu[6]&0xf0 != 0 {
return nil
}
rest := tpdu[x224FixedSize:]
m := analyzer.PropMap{}
if code == x224CodeConnectionRequest {
// Both the cookie ("Cookie: mstshash=<user>") and the load balancer
// routing token ("Cookie: msts=<token>") are terminated by CRLF.
if bytes.HasPrefix(rest, rdpCookiePrefix) {
i := bytes.Index(rest, []byte("\r\n"))
if i < 0 {
return nil
}
m["cookie"] = string(rest[len(rdpCookiePrefix):i])
rest = rest[i+2:]
}
// A client that sends no RDP_NEG_REQ can only do standard RDP
// security (PROTOCOL_RDP).
m["protocols"] = 0
} else {
m["protocol"] = 0
}
if len(rest) == 0 {
return m
}
// Anything after the negotiation structure, such as the correlation info
// a client may append, is ignored.
if len(rest) < rdpNegSize || binary.LittleEndian.Uint16(rest[2:4]) != rdpNegSize {
return nil
}
v := int(binary.LittleEndian.Uint32(rest[4:8]))
switch {
case code == x224CodeConnectionRequest && rest[0] == rdpNegReq:
m["protocols"] = v
case code == x224CodeConnectionConfirm && rest[0] == rdpNegRsp:
m["protocol"] = v
case code == x224CodeConnectionConfirm && rest[0] == rdpNegFailure:
delete(m, "protocol")
m["failure"] = v
default:
return nil
}
return m
}
+131
View File
@@ -0,0 +1,131 @@
package tcp
import (
"reflect"
"testing"
"github.com/apernet/OpenGFW/analyzer"
)
// rdpConnectionRequest is what mstsc sends first: a cookie with the user name
// and an RDP_NEG_REQ asking for PROTOCOL_SSL | PROTOCOL_HYBRID |
// PROTOCOL_HYBRID_EX.
var rdpConnectionRequest = []byte{
0x03, 0x00, 0x00, 0x2c, // TPKT, length 44
0x27, 0xe0, 0x00, 0x00, 0x00, 0x00, 0x00, // X.224 CR
'C', 'o', 'o', 'k', 'i', 'e', ':', ' ', 'm', 's', 't', 's', 'h', 'a', 's', 'h', '=', 'e', 'l', 't', 'o', 'n', 's', '\r', '\n',
0x01, 0x00, 0x08, 0x00, 0x0b, 0x00, 0x00, 0x00, // RDP_NEG_REQ
}
func rdpConnectionConfirm(negType byte, v byte) []byte {
return []byte{
0x03, 0x00, 0x00, 0x13, // TPKT, length 19
0x0e, 0xd0, 0x00, 0x00, 0x12, 0x34, 0x00, // X.224 CC
negType, 0x00, 0x08, 0x00, v, 0x00, 0x00, 0x00,
}
}
var rdpTLSClientHelloProps = analyzer.PropMap{"cookie": "mstshash=eltons", "protocols": 11}
func TestTlsRDPClientHello(t *testing.T) {
data := append(append([]byte{}, rdpConnectionRequest...), tlsExampleClientHello...)
for name, segs := range map[string][][]byte{
"one segment": {data},
"byte by byte": splitEvery(data, 1),
"pdu, hello": {rdpConnectionRequest, tlsExampleClientHello},
} {
props, _ := feedTLS(newTLSStream(nil), false, segs...)
if got := props.Get("rdp_req"); !reflect.DeepEqual(got, rdpTLSClientHelloProps) {
t.Errorf("%s: rdp_req = %v, want %v", name, got, rdpTLSClientHelloProps)
}
if got := props.Get("req"); !reflect.DeepEqual(got, tlsExampleClientHelloProps) {
t.Errorf("%s: req = %v, want %v", name, got, tlsExampleClientHelloProps)
}
}
}
// The negotiation is published before any TLS shows up, so that rules can
// already see the stream is RDP.
func TestTlsRDPNegotiationReportedEarly(t *testing.T) {
s := newTLSStream(nil)
props, done := feedTLS(s, false, rdpConnectionRequest)
if done {
t.Fatal("stream done after connection request")
}
if got := props.Get("rdp_req"); !reflect.DeepEqual(got, rdpTLSClientHelloProps) {
t.Errorf("rdp_req = %v, want %v", got, rdpTLSClientHelloProps)
}
props, done = feedTLS(s, true, rdpConnectionConfirm(0x02, 0x02))
if done {
t.Fatal("stream done after connection confirm")
}
if got, want := props.Get("rdp_resp"), (analyzer.PropMap{"protocol": 2}); !reflect.DeepEqual(got, want) {
t.Errorf("rdp_resp = %v, want %v", got, want)
}
}
func TestTlsRDPConnectionConfirm(t *testing.T) {
for name, tc := range map[string]struct {
pdu []byte
want analyzer.PropMap
}{
"ssl": {rdpConnectionConfirm(0x02, 0x01), analyzer.PropMap{"protocol": 1}},
"hybrid_ex": {rdpConnectionConfirm(0x02, 0x08), analyzer.PropMap{"protocol": 8}},
"failure": {rdpConnectionConfirm(0x03, 0x05), analyzer.PropMap{"failure": 5}},
"no neg rsp": {[]byte{0x03, 0x00, 0x00, 0x0b, 0x06, 0xd0, 0x00, 0x00, 0x12, 0x34, 0x00}, analyzer.PropMap{"protocol": 0}},
} {
props, _ := feedTLS(newTLSStream(nil), true, tc.pdu)
if got := props.Get("rdp_resp"); !reflect.DeepEqual(got, tc.want) {
t.Errorf("%s: rdp_resp = %v, want %v", name, got, tc.want)
}
}
}
// Standard RDP security has no TLS: the client goes on with MCS, which is not
// a TLS record, so the analyzer stops with only the negotiation published.
func TestTlsRDPWithoutTLS(t *testing.T) {
mcs := []byte{0x03, 0x00, 0x01, 0xa0, 0x02, 0xf0, 0x80, 0x7f, 0x65}
props, done := feedTLS(newTLSStream(nil), false, rdpConnectionRequest, mcs)
if !done {
t.Error("stream not done")
}
if props.Get("rdp_req") == nil || props.Get("req") != nil {
t.Errorf("props = %v, want rdp_req only", props)
}
}
// truncatedNegReq is rdpConnectionRequest with the last two bytes of the
// RDP_NEG_REQ cut off and the lengths adjusted to match.
func truncatedNegReq() []byte {
pdu := append([]byte{}, rdpConnectionRequest[:len(rdpConnectionRequest)-2]...)
pdu[3] -= 2
pdu[4] -= 2
return pdu
}
func TestTlsRDPRejectsMalformedPrelude(t *testing.T) {
mutate := func(i int, b byte) []byte {
pdu := append([]byte{}, rdpConnectionRequest...)
pdu[i] = b
return pdu
}
for name, pdu := range map[string][]byte{
"reserved set": mutate(1, 0x01),
"wrong code": mutate(5, 0xd0),
"length mismatch": mutate(4, 0x26),
"class 2": mutate(10, 0x20),
"cookie w/o crlf": mutate(len(rdpConnectionRequest)-10, 'x'),
"bad neg type": mutate(len(rdpConnectionRequest)-8, 0x02),
"tpkt too short": {0x03, 0x00, 0x00, 0x05, 0x00},
"not tpkt nor tls": {0x05, 0x01, 0x00},
"truncated neg req": truncatedNegReq(),
} {
props, done := feedTLS(newTLSStream(nil), false, pdu)
if !done {
t.Errorf("%s: stream not done", name)
}
if len(props) != 0 {
t.Errorf("%s: props = %v, want none", name, props)
}
}
}
+150 -31
View File
@@ -7,39 +7,42 @@ import (
"github.com/apernet/OpenGFW/analyzer"
)
func TestTlsStreamParsing_ClientHello(t *testing.T) {
// example packet taken from <https://tls12.xargs.org/#client-hello/annotated>
clientHello := []byte{
0x16, 0x03, 0x01, 0x00, 0xa5, 0x01, 0x00, 0x00, 0xa1, 0x03, 0x03, 0x00,
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c,
0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x00, 0x00, 0x20, 0xcc, 0xa8,
0xcc, 0xa9, 0xc0, 0x2f, 0xc0, 0x30, 0xc0, 0x2b, 0xc0, 0x2c, 0xc0, 0x13,
0xc0, 0x09, 0xc0, 0x14, 0xc0, 0x0a, 0x00, 0x9c, 0x00, 0x9d, 0x00, 0x2f,
0x00, 0x35, 0xc0, 0x12, 0x00, 0x0a, 0x01, 0x00, 0x00, 0x58, 0x00, 0x00,
0x00, 0x18, 0x00, 0x16, 0x00, 0x00, 0x13, 0x65, 0x78, 0x61, 0x6d, 0x70,
0x6c, 0x65, 0x2e, 0x75, 0x6c, 0x66, 0x68, 0x65, 0x69, 0x6d, 0x2e, 0x6e,
0x65, 0x74, 0x00, 0x05, 0x00, 0x05, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00,
0x0a, 0x00, 0x0a, 0x00, 0x08, 0x00, 0x1d, 0x00, 0x17, 0x00, 0x18, 0x00,
0x19, 0x00, 0x0b, 0x00, 0x02, 0x01, 0x00, 0x00, 0x0d, 0x00, 0x12, 0x00,
0x10, 0x04, 0x01, 0x04, 0x03, 0x05, 0x01, 0x05, 0x03, 0x06, 0x01, 0x06,
0x03, 0x02, 0x01, 0x02, 0x03, 0xff, 0x01, 0x00, 0x01, 0x00, 0x00, 0x12,
0x00, 0x00,
}
want := analyzer.PropMap{
"ciphers": []uint16{52392, 52393, 49199, 49200, 49195, 49196, 49171, 49161, 49172, 49162, 156, 157, 47, 53, 49170, 10},
"compression": []uint8{0},
"random": []uint8{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31},
"session": []uint8{},
"sni": "example.ulfheim.net",
"version": uint16(771),
}
// tlsExampleClientHello is taken from <https://tls12.xargs.org/#client-hello/annotated>.
var tlsExampleClientHello = []byte{
0x16, 0x03, 0x01, 0x00, 0xa5, 0x01, 0x00, 0x00, 0xa1, 0x03, 0x03, 0x00,
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c,
0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x00, 0x00, 0x20, 0xcc, 0xa8,
0xcc, 0xa9, 0xc0, 0x2f, 0xc0, 0x30, 0xc0, 0x2b, 0xc0, 0x2c, 0xc0, 0x13,
0xc0, 0x09, 0xc0, 0x14, 0xc0, 0x0a, 0x00, 0x9c, 0x00, 0x9d, 0x00, 0x2f,
0x00, 0x35, 0xc0, 0x12, 0x00, 0x0a, 0x01, 0x00, 0x00, 0x58, 0x00, 0x00,
0x00, 0x18, 0x00, 0x16, 0x00, 0x00, 0x13, 0x65, 0x78, 0x61, 0x6d, 0x70,
0x6c, 0x65, 0x2e, 0x75, 0x6c, 0x66, 0x68, 0x65, 0x69, 0x6d, 0x2e, 0x6e,
0x65, 0x74, 0x00, 0x05, 0x00, 0x05, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00,
0x0a, 0x00, 0x0a, 0x00, 0x08, 0x00, 0x1d, 0x00, 0x17, 0x00, 0x18, 0x00,
0x19, 0x00, 0x0b, 0x00, 0x02, 0x01, 0x00, 0x00, 0x0d, 0x00, 0x12, 0x00,
0x10, 0x04, 0x01, 0x04, 0x03, 0x05, 0x01, 0x05, 0x03, 0x06, 0x01, 0x06,
0x03, 0x02, 0x01, 0x02, 0x03, 0xff, 0x01, 0x00, 0x01, 0x00, 0x00, 0x12,
0x00, 0x00,
}
var tlsExampleClientHelloProps = analyzer.PropMap{
"ciphers": []uint16{52392, 52393, 49199, 49200, 49195, 49196, 49171, 49161, 49172, 49162, 156, 157, 47, 53, 49170, 10},
"compression": []uint8{0},
"random": []uint8{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31},
"session": []uint8{},
"sni": "example.ulfheim.net",
"version": uint16(771),
}
func TestTlsStreamParsing_ClientHello(t *testing.T) {
s := newTLSStream(nil)
u, _ := s.Feed(false, false, false, 0, clientHello)
got := u.M.Get("req")
if !reflect.DeepEqual(got, want) {
t.Errorf("%d B parsed = %v, want %v", len(clientHello), got, want)
u, _ := s.Feed(false, false, false, 0, tlsExampleClientHello)
if got := u.M.Get("req"); !reflect.DeepEqual(got, tlsExampleClientHelloProps) {
t.Errorf("parsed = %v, want %v", got, tlsExampleClientHelloProps)
}
if got := u.M.Get("req_records"); got != 1 {
t.Errorf("req_records = %v, want 1", got)
}
}
@@ -67,3 +70,119 @@ func TestTlsStreamParsing_ServerHello(t *testing.T) {
t.Errorf("%d B parsed = %v, want %v", len(serverHello), got, want)
}
}
// tlsFragment re-encodes the handshake message in a single-record hello as
// records carrying at most size bytes of it each.
func tlsFragment(hello []byte, size int) []byte {
var out []byte
for msg := hello[5:]; len(msg) > 0; {
n := min(size, len(msg))
out = append(out, hello[0], hello[1], hello[2], byte(n>>8), byte(n))
out = append(out, msg[:n]...)
msg = msg[n:]
}
return out
}
// feedTLS feeds each segment in turn and returns the properties as the engine
// would have merged them, along with whether the stream said it was done.
func feedTLS(s *tlsStream, rev bool, segs ...[]byte) (analyzer.PropMap, bool) {
props := analyzer.PropMap{}
done := false
for _, seg := range segs {
var u *analyzer.PropUpdate
u, done = s.Feed(rev, false, false, 0, seg)
if u != nil {
for k, v := range u.M {
props[k] = v
}
}
}
return props, done
}
// Splitting the ClientHello across records must not hide it: the parse result
// has to be the same wherever the record boundaries fall.
func TestTlsClientHelloRecordFragmentation(t *testing.T) {
msgLen := len(tlsExampleClientHello) - 5
for _, size := range []int{1, 2, 3, 4, 5, 9, 50, msgLen - 1} {
data := tlsFragment(tlsExampleClientHello, size)
wantRecords := (msgLen + size - 1) / size
for name, segs := range map[string][][]byte{
"one segment": {data},
"byte by byte": splitEvery(data, 1),
"7 byte segs": splitEvery(data, 7),
} {
props, _ := feedTLS(newTLSStream(nil), false, segs...)
if got := props.Get("req"); !reflect.DeepEqual(got, tlsExampleClientHelloProps) {
t.Errorf("size %d, %s: parsed = %v, want %v", size, name, got, tlsExampleClientHelloProps)
}
if got := props.Get("req_records"); got != wantRecords {
t.Errorf("size %d, %s: req_records = %v, want %d", size, name, got, wantRecords)
}
}
}
}
// Fragmentation is published before the hello is complete, so that a rule can
// act on it even if the hello never completes.
func TestTlsClientHelloFragmentationReportedEarly(t *testing.T) {
data := tlsFragment(tlsExampleClientHello, 1)
s := newTLSStream(nil)
props, done := feedTLS(s, false, data[:12])
if done {
t.Fatal("stream done after two records")
}
if props.Get("req") != nil {
t.Errorf("req published early: %v", props.Get("req"))
}
if got := props.Get("req_records"); got != 2 {
t.Errorf("req_records = %v, want 2", got)
}
// Further records are not published one by one...
if u, _ := s.Feed(false, false, false, 0, data[12:60]); u != nil {
t.Errorf("unexpected update %v", u.M)
}
// ...but the count reached is, if the stream gets cut off.
u := s.Close(true)
if u == nil {
t.Fatal("no update on close")
}
if got := u.M.Get("req_records"); got != 10 {
t.Errorf("req_records on close = %v, want 10", got)
}
}
func TestTlsClientHelloRejectsInterleavedRecords(t *testing.T) {
data := tlsFragment(tlsExampleClientHello, 50)
// Turn the second record into an alert.
data[5+50] = 0x15
props, done := feedTLS(newTLSStream(nil), false, data)
if !done {
t.Error("stream not done")
}
if props.Get("req") != nil {
t.Errorf("req = %v, want nil", props.Get("req"))
}
}
func TestTlsStreamIgnoresUnfragmentedPartialHelloOnClose(t *testing.T) {
s := newTLSStream(nil)
if props, _ := feedTLS(s, false, tlsExampleClientHello[:100]); len(props) != 0 {
t.Errorf("props = %v, want none", props)
}
if u := s.Close(true); u != nil {
t.Errorf("close update = %v, want nil", u.M)
}
}
func splitEvery(data []byte, n int) [][]byte {
var segs [][]byte
for len(data) > 0 {
k := min(n, len(data))
segs = append(segs, data[:k])
data = data[k:]
}
return segs
}
+93
View File
@@ -0,0 +1,93 @@
# 拦截 TLS、放行 RDP
## 问题
RDP 不是一连上就开始 TLS:客户端先在 TPKT 里发一个 X.224 Connection Request,服务端回一个 Connection Confirm,双方借此协商安全协议([MS-RDPBCGR] 1.3.1.1),协商完才在同一条连接上开始 TLS 握手。
`tls` 分析器以前要求流的第一个字节就是 TLS record(`0x16`),碰到 RDP 开头的 `03 00` 直接放弃。结果是:
- RDP 里的 TLS 对 `tls` 规则完全不可见,没法按 RDP 单独写规则;
- 反过来,任何流量只要在 TLS 前面垫一段假的 X.224 协商,就能绕过"拦截所有 TLS"的规则。
## 做法
`tls` 分析器现在把 RDP 协商当作 hello 之前可选的一段前导来解析:
- 第一个字节是 `0x16`:和以前一样,直接当 TLS 解析。
- 第一个字节是 `0x03`:按 TPKT + X.224 解析。客户端方向要求是 Connection Request,服务端方向要求是 Connection Confirm;长度、class、协商结构任何一处不对都视为非法,分析器取消。
- 前导解析完后照常解析 ClientHello / ServerHello,record 分片重组等逻辑完全一样。
协商一解析出来就上报,不用等 TLS:
| 属性 | 类型 | 说明 |
| --- | --- | --- |
| `tls.rdp_req.cookie` | string | Connection Request 里的 cookie,去掉 `Cookie: ` 前缀和 CRLF,如 `mstshash=alice`;没有 cookie 时不存在 |
| `tls.rdp_req.protocols` | int | RDP_NEG_REQ 里客户端请求的协议位图;没有 RDP_NEG_REQ 时为 0 |
| `tls.rdp_resp.protocol` | int | RDP_NEG_RSP 里服务端选定的协议;没有 RDP_NEG_RSP 时为 0 |
| `tls.rdp_resp.failure` | int | 服务端回的是 RDP_NEG_FAILURE 时的失败码(此时没有 `protocol`) |
协议取值:
| 值 | 名称 | 是否走 TLS |
| --- | --- | --- |
| 0 | PROTOCOL_RDP(标准 RDP 安全,RC4) | 否 |
| 1 | PROTOCOL_SSL | 是 |
| 2 | PROTOCOL_HYBRID(CredSSP / NLA) | 是 |
| 4 | PROTOCOL_RDSTLS | 是 |
| 8 | PROTOCOL_HYBRID_EX | 是 |
| 16 | PROTOCOL_RDSAAD | 是 |
## 规则
规则按顺序匹配、第一条命中的生效,所以放行 RDP 的规则要写在拦截 TLS 的前面:
```yaml
- name: allow-rdp-tls
action: allow
log: true
expr: >
tls != nil && tls.req != nil &&
tls.rdp_req != nil && tls.rdp_resp != nil &&
tls.rdp_resp.protocol in [1, 2, 4, 8, 16]
- name: block-tls
action: block
log: true
expr: tls != nil && tls.req != nil
```
放行条件同时要求:
- **`tls.req != nil`**:ClientHello 已经完整解析。只看到协商就放行的话,协商之后跟什么都行。
- **`tls.rdp_resp != nil` 且 `protocol` 是走 TLS 的协议**:真实的 RDP 只有在服务端选了 TLS 类协议之后才会发 ClientHello。服务端选了 PROTOCOL_RDP 或者回了协商失败,后面却出现 TLS,说明是伪造的,交给 `block-tls` 拦掉。
正常 RDP 的时序是 CR → CC → ClientHello,ClientHello 到的时候 `rdp_resp` 早已上报,不存在规则先看到 `req` 再看到 `rdp_resp` 的竞争。
标准 RDP 安全(`protocol == 0`)的连接里根本没有 TLS,两条规则都不会命中,按默认放行。
### 收紧放行条件
上面的规则只验证"协商格式正确、协商结果和后续 TLS 一致"。客户端和服务端串通的话,完全可以按规范发一套协商再跑任意 TLS,所以按需叠加:
```yaml
# 只放行标准端口
expr: ... && port.dst == 3389
# 只放行指定服务器
expr: ... && ip.dst in ["10.0.0.10", "10.0.0.11"]
# 要求客户端带 mstshash cookie(mstsc 通常会带,但不保证,先观察日志再启用)
expr: ... && tls.rdp_req.cookie != nil && tls.rdp_req.cookie startsWith "mstshash="
```
## 行为变化
**以前按 `tls != nil && tls.req != nil` 写的规则现在会命中 RDP。** 以前 RDP 里的 TLS 看不到,所以不受影响;现在看得到了。不想拦 RDP 的话,必须像上面那样在前面加一条放行规则。
## 局限
**只认 RDP 这一种前导。** 其他协议里的 STARTTLS(SMTP、IMAP、XMPP 等)和任何非 TLS、非 TPKT 开头的前缀仍然会让 `tls` 分析器直接放弃。要做到"拦截所有 TLS",还需要配合 `fet` 等不依赖协议开头的检测。
**RD Gateway 不在此列。** 经由 RD 网关的 RDP 跑在 443 端口的 HTTPS 上,开头就是普通 TLS,没有 X.224 协商,只能按 SNI / IP 放行。
**RDP 的 UDP 传输不在此列。** RDP-UDP 用的是 DTLS,走 UDP,与 `tls` 分析器无关。
+1 -1
View File
@@ -28,7 +28,7 @@ table inet filter {
- **快路径**:首个分段本身就够长时(正常流量的首包都是几百字节),直接在原始切片上分类,不拷贝、不分配,开销与改动前完全一致。只有小于 16 字节的首段才会走缓冲。
- **段数上限**:最多等 `fetMaxSegments`(4)个分段。否则一个只发 1 字节然后装死的对端,就能让流一直留在活跃状态、拿不到 connmark 快速通道,直到 TCP 超时(默认 10 分钟)为止——那会变成一个可被主动利用的资源消耗面。
`http`、`tls`、`minecraft` 等分析器本来就用 `utils.ByteBuffer` 做增量解析,不受这个手法影响。
`http`、`tls`、`minecraft` 等分析器本来就用 `utils.ByteBuffer` 做增量解析,不受这个手法影响。注意这只针对 TCP 分段;TLS 自己记录层上的分片是另一回事,见 [TLS record 分片规避与检测](tls-record-fragmentation.zh.md)。
## 第二层防御:把窗口改写本身当指纹
+70
View File
@@ -0,0 +1,70 @@
# TLS record 分片规避与检测
## 规避手法
TLS 允许一个握手消息跨任意多个 record 传输(RFC 8446 §5.1)。客户端把 ClientHello 拆成很多个小 record,服务端照常把它们拼回去完成握手;但只看第一个 record 的 DPI 就拿不到完整的 ClientHello,SNI 也就看不到了。Xray 的 `fragment`(`packets: "tlshello"`)以及各类"TLS 分片"工具用的都是这个手法。
它和 [TCP 窗口操纵](tcp-window-evasion.zh.md) 是两层不同的东西:TCP 分段切的是字节流,OpenGFW 的 TCP 重组会把它抹平;record 分片切的是 TLS 自己的记录层,重组后的字节流里每一片前面都还带着 5 字节的 record 头。
`tls` 分析器以前默认 ClientHello 只在一个 record 里,读完第一个 record 头就直接按握手长度切数据,碰到分片会:
- 第一个 record 少于 4 字节握手数据时,把下一个 record 的头误当成握手长度(例如 `16 03 01` 被读成 0x160301),一直等到 8192 字节的上限;
- 其他情况下,后续 record 的头混进消息体导致解析失败,分析器取消。
两种情况都不产出任何属性,所有基于 `tls.req.sni` 的规则都会被绕过。
## 第一层防御:在 record 层重组
现在分析器先剥掉 record 层,把各个 record 的内容拼回完整的握手消息再解析,所以无论切在哪里、切成多少片,`tls.req` 的解析结果都和不分片时完全一样。ServerHello 方向也做了同样的处理。
- 握手消息中间夹了其他类型的 record(alert、change_cipher_spec 等)会被视为非法,分析器取消。
- record 长度超过 2^14 视为非法。
- 空 record 按规范不允许出现在握手中,但这里不拒绝,只是计数——拒绝的话反而给了对方一个让分析中止的开关。
- 常见情况下整个 ClientHello 在一个 record 里,此时直接引用原始缓冲区,不做额外拷贝。
## 第二层防御:把分片本身当指纹
| 属性 | 类型 | 说明 |
| --- | --- | --- |
| `tls.req_records` | int | ClientHello 跨了几个 record |
| `tls.resp_records` | int | ServerHello 跨了几个 record |
主流浏览器和 TLS 库都把 ClientHello 放在一个 record 里,即使是带后量子密钥交换、将近 2KB 的 ClientHello 也远小于 16KB 的 record 上限。所以 `req_records > 1` 本身就是很强的规避信号,误报率很低。
这个计数放在 `req` 外面,是为了保证 `tls.req != nil` 仍然表示"ClientHello 已完整解析",现有规则的语义不变。
### 提前上报
分片一出现(第二个 record 开始)就会上报一次 `req_records`,不用等消息收完。原因是分析器有 8192 字节的 `Limit()`:切成每片 1 字节时,每个字节要额外带 5 字节的 record 头,一个 1.8KB 的 ClientHello 会膨胀到 11KB 左右,永远收不完。所以规则不能依赖 SNI 才动手。
之后的 record 不会逐个上报(否则每片都要跑一次规则匹配);消息收完时,或者分析器因为超限被关闭时,再上报最终的计数。所以规则里看到的值可能是 2,也可能是最终值——写阈值时按 `> 1` 来写。
### 规则示例
直接拦截分片的 ClientHello:
```yaml
- name: tls-record-fragmentation
action: block
log: true
expr: tls != nil && tls.req_records != nil && tls.req_records > 1
```
> [!IMPORTANT]
> 要判断 `tls.req_records != nil`,不能只判 `tls != nil`:只看到服务端方向时,`tls` 里可能只有 `resp` 相关的字段。
按 SNI 拦截的规则不需要任何改动,分片后照样能匹配:
```yaml
- name: block-example
action: block
expr: tls != nil && tls.req != nil && tls.req.sni endsWith "example.com"
```
## 局限
**只统计 hello 消息本身。** ClientHello 收完就停止计数,后面的 record 不影响 `req_records`。
**超长的分片拿不到 SNI。** 分片把 ClientHello 撑过 8192 字节时,`tls.req` 不会出现,只能靠 `req_records` 拦截。需要同时拿到 SNI 的话可以调大 `Limit()`,代价是每条 TLS 连接的缓冲上限相应变大。
**QUIC 不在此列。** QUIC 的 ClientHello 在 CRYPTO 帧里,走的是 `quic` 分析器,与这里无关。