lbcd/rpcwebsocket.go

1428 lines
43 KiB
Go
Raw Normal View History

2014-01-01 17:16:15 +01:00
// Copyright (c) 2013-2014 Conformal Systems LLC.
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package main
import (
"bytes"
"code.google.com/p/go.net/websocket"
"container/list"
"crypto/sha256"
"crypto/subtle"
"encoding/base64"
"encoding/hex"
"encoding/json"
2014-02-24 15:10:59 +01:00
"errors"
"fmt"
"github.com/conformal/btcjson"
"github.com/conformal/btcscript"
"github.com/conformal/btcutil"
"github.com/conformal/btcwire"
"github.com/conformal/btcws"
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
"io"
"sync"
"time"
)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
const (
// websocketSendBufferSize is the number of elements the send channel
// can queue before blocking. Note that this only applies to requests
// handled directly in the websocket client input handler or the async
// handler since notifications have their own queueing mechanism
// independent of the send channel buffer.
websocketSendBufferSize = 50
)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// timeZeroVal is simply the zero value for a time.Time and is used to avoid
// creating multiple instances.
var timeZeroVal time.Time
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// wsCommandHandler describes a callback function used to handle a specific
// command.
type wsCommandHandler func(*wsClient, btcjson.Cmd) (interface{}, *btcjson.Error)
// wsHandlers maps RPC command strings to appropriate websocket handler
// functions.
var wsHandlers = map[string]wsCommandHandler{
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
"getbestblock": handleGetBestBlock,
"getcurrentnet": handleGetCurrentNet,
"notifyblocks": handleNotifyBlocks,
"notifyallnewtxs": handleNotifyAllNewTXs,
"notifynewtxs": handleNotifyNewTXs,
"notifyspent": handleNotifySpent,
"rescan": handleRescan,
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// wsAsyncHandlers holds the websocket commands which should be run
// asynchronously to the main input handler goroutine. This allows long-running
// operations to run concurrently (and one at a time) while still responding
// to the majority of normal requests which can be answered quickly.
var wsAsyncHandlers = map[string]bool{
"rescan": true,
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// WebsocketHandler handles a new websocket client by creating a new wsClient,
// starting it, and blocking until the connection closes. Since it blocks, it
// must be run in a separate goroutine. It should be invoked from the websocket
// server handler which runs each new connection in a new goroutine thereby
// satisfying the requirement.
func (s *rpcServer) WebsocketHandler(conn *websocket.Conn, remoteAddr string,
authenticated bool) {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Clear the read deadline that was set before the websocket hijacked
// the connection.
conn.SetReadDeadline(timeZeroVal)
// Limit max number of websocket clients.
rpcsLog.Infof("New websocket client %s", remoteAddr)
if s.ntfnMgr.NumClients()+1 > cfg.RPCMaxWebsockets {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
rpcsLog.Infof("Max websocket clients exceeded [%d] - "+
"disconnecting client %s", cfg.RPCMaxWebsockets,
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
remoteAddr)
conn.Close()
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Create a new websocket client to handle the new websocket connection
// and wait for it to shutdown. Once it has shutdown (and hence
// disconnected), remove it and any notifications it registered for.
client := newWebsocketClient(s, conn, remoteAddr, authenticated)
s.ntfnMgr.AddClient(client)
client.Start()
client.WaitForShutdown()
s.ntfnMgr.RemoveClient(client)
rpcsLog.Infof("Disconnected websocket client %s", remoteAddr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// wsNotificationManager is a connection and notification manager used for
// websockets. It allows websocket clients to register for notifications they
// are interested in. When an event happens elsewhere in the code such as
// transactions being added to the memory pool or block connects/disconnects,
// the notification manager is provided with the relevant details needed to
// figure out which websocket clients need to be notified based on what they
// have registered for and notifies them accordingly. It is also used to keep
// track of all connected websocket clients.
type wsNotificationManager struct {
sync.Mutex
// server is the RPC server the notification manager is associated with.
server *rpcServer
// clients is a map of all currently connected websocket clients.
clients map[chan bool]*wsClient
// Maps used to hold lists of websocket clients to be notified on
// certain events. Each websocket client also keeps maps for the events
// which have multiple triggers to make removal from these lists on
// connection close less horrendously expensive.
blockNotifications map[chan bool]*wsClient
txNotifications map[chan bool]*wsClient
spentNotifications map[btcwire.OutPoint]map[chan bool]*wsClient
addrNotifications map[string]map[chan bool]*wsClient
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// NumClients returns the number of clients actively being served.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) NumClients() int {
m.Lock()
defer m.Unlock()
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
return len(m.clients)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// AddBlockUpdateRequest requests block update notifications to the passed
// websocket client.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) AddBlockUpdateRequest(wsc *wsClient) {
m.Lock()
defer m.Unlock()
// Add the client to the map to notify when block updates are seen.
// Use the quit channel as a unique id for the client since it is quite
// a bit more efficient than using the entire struct.
m.blockNotifications[wsc.quit] = wsc
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// RemoveBlockUpdateRequest removes block update notifications for the passed
// websocket client.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) RemoveBlockUpdateRequest(wsc *wsClient) {
m.Lock()
defer m.Unlock()
// Delete the client from the map to notify when block updates are seen.
// Use the quit channel as a unique id for the client since it is quite
// a bit more efficient than using the entire struct.
delete(m.blockNotifications, wsc.quit)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// NotifyBlockConnected notifies websocket clients that have registered for
// block updates when a block is connected to the main chain.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) NotifyBlockConnected(block *btcutil.Block) {
m.Lock()
defer m.Unlock()
// Nothing to do if there are no websocket clients registered to
// receive notifications that result from a newly connected block.
if len(m.blockNotifications) == 0 {
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
hash, err := block.Sha()
if err != nil {
rpcsLog.Error("Bad block; connected block notification dropped")
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Notify interested websocket clients about the connected block.
ntfn := btcws.NewBlockConnectedNtfn(hash.String(), int32(block.Height()))
marshalledJSON, err := json.Marshal(ntfn)
if err != nil {
rpcsLog.Error("Failed to marshal block connected notification: "+
"%v", err)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, wsc := range m.blockNotifications {
wsc.QueueNotification(marshalledJSON)
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// NotifyBlockDisconnected notifies websocket clients that have registered for
// block updates when a block is disconnected from the main chain (due to a
// reorganize).
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) NotifyBlockDisconnected(block *btcutil.Block) {
m.Lock()
defer m.Unlock()
// Nothing to do if there are no websocket clients registered to
// receive notifications that result from a newly connected block.
if len(m.blockNotifications) == 0 {
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
hash, err := block.Sha()
if err != nil {
rpcsLog.Error("Bad block; disconnected block notification " +
"dropped")
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Notify interested websocket clients about the disconnected block.
ntfn := btcws.NewBlockDisconnectedNtfn(hash.String(),
int32(block.Height()))
marshalledJSON, err := json.Marshal(ntfn)
if err != nil {
rpcsLog.Error("Failed to marshal block disconnected "+
"notification: %v", err)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, wsc := range m.blockNotifications {
wsc.QueueNotification(marshalledJSON)
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// AddNewTxRequest requests notifications to the passed websocket client when
// new transactions are added to the memory pool.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) AddNewTxRequest(wsc *wsClient) {
m.Lock()
defer m.Unlock()
// Add the client to the map to notify when a new transaction is added
// to the memory pool. Use the quit channel as a unique id for the
// client since it is quite a bit more efficient than using the entire
// struct.
m.txNotifications[wsc.quit] = wsc
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// RemoveNewTxRequest removes notifications to the passed websocket client when
// new transaction are added to the memory pool.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) RemoveNewTxRequest(wsc *wsClient) {
m.Lock()
defer m.Unlock()
// Delete the client from the map to notify when a new transaction is
// seen in the memory pool. Use the quit channel as a unique id for the
// client since it is quite a bit more efficient than using the entire
// struct.
delete(m.txNotifications, wsc.quit)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// NotifyForNewTx notifies websocket clients that have registerd for updates
// when a new transaction is added to the memory pool.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) NotifyForNewTx(tx *btcutil.Tx) {
m.Lock()
defer m.Unlock()
// Nothing to do if there are no websocket clients registered to
// receive notifications about transactions added to the memory pool.
if len(m.txNotifications) == 0 {
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
txID := tx.Sha().String()
mtx := tx.MsgTx()
var amount int64
for _, txOut := range mtx.TxOut {
amount += txOut.Value
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
ntfn := btcws.NewAllTxNtfn(txID, amount)
marshalledJSON, err := json.Marshal(ntfn)
if err != nil {
rpcsLog.Errorf("Failed to marshal tx notification: %s", err.Error())
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
var verboseNtfn *btcws.AllVerboseTxNtfn
var marshalledJSONVerbose []byte
for _, wsc := range m.txNotifications {
if wsc.verboseTxUpdates {
if verboseNtfn == nil {
rawTx, err := createTxRawResult(m.server.server.btcnet, txID, mtx, nil, 0, nil)
if err != nil {
return
}
verboseNtfn = btcws.NewAllVerboseTxNtfn(rawTx)
marshalledJSONVerbose, err = json.Marshal(verboseNtfn)
if err != nil {
rpcsLog.Errorf("Failed to marshal verbose tx notification: %s", err.Error())
return
}
}
wsc.QueueNotification(marshalledJSONVerbose)
} else {
wsc.QueueNotification(marshalledJSON)
}
}
}
2014-02-24 15:10:59 +01:00
// addSpentRequest is the internal function which implements the public
// AddSpentRequest. See the comment for AddSpentRequest for more details.
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
//
2014-02-24 15:10:59 +01:00
// This function MUST be called with the notification manager lock held.
func (m *wsNotificationManager) addSpentRequest(wsc *wsClient, op *btcwire.OutPoint) {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Track the request in the client as well so it can be quickly be
// removed on disconnect.
wsc.spentRequests[*op] = struct{}{}
// Add the client to the list to notify when the outpoint is seen.
// Create the list as needed.
cmap, ok := m.spentNotifications[*op]
if !ok {
cmap = make(map[chan bool]*wsClient)
m.spentNotifications[*op] = cmap
}
cmap[wsc.quit] = wsc
}
2014-02-24 15:10:59 +01:00
// AddSpentRequest requests an notification when the passed outpoint is
// confirmed spent (contained in a block connected to the main chain) for the
// passed websocket client. The request is automatically removed once the
// notification has been sent.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) AddSpentRequest(wsc *wsClient, op *btcwire.OutPoint) {
m.Lock()
defer m.Unlock()
m.addSpentRequest(wsc, op)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// removeSpentRequest is the internal function which implements the public
// RemoveSpentRequest. See the comment for RemoveSpentRequest for more details.
//
// This function MUST be called with the notification manager lock held.
func (m *wsNotificationManager) removeSpentRequest(wsc *wsClient, op *btcwire.OutPoint) {
// Remove the request tracking from the client.
delete(wsc.spentRequests, *op)
// Remove the client from the list to notify.
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
notifyMap, ok := m.spentNotifications[*op]
if !ok {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
rpcsLog.Warnf("Attempt to remove nonexistent spent request "+
"for websocket client %s", wsc.addr)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
delete(notifyMap, wsc.quit)
// Remove the map entry altogether if there are no more clients
// interested in it.
if len(notifyMap) == 0 {
delete(m.spentNotifications, *op)
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// RemoveSpentRequest removes a request from the passed websocket client to be
// notified when the passed outpoint is confirmed spent (contained in a block
// connected to the main chain).
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) RemoveSpentRequest(wsc *wsClient, op *btcwire.OutPoint) {
m.Lock()
defer m.Unlock()
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
m.removeSpentRequest(wsc, op)
}
2014-02-24 15:10:59 +01:00
// txHexString returns the serialized transaction encoded in hexadecimal.
func txHexString(tx *btcutil.Tx) string {
var buf bytes.Buffer
// Ignore Serialize's error, as writing to a bytes.buffer cannot fail.
tx.MsgTx().Serialize(&buf)
return hex.EncodeToString(buf.Bytes())
}
// notifyForTxOuts examines each transaction output, notifying interested
// websocket clients of the transaction if an output spends to a watched
// address. A spent notification request is automatically registered for
// the client for each matching output.
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
//
// This function MUST be called with the notification manager lock held.
func (m *wsNotificationManager) notifyForTxOuts(tx *btcutil.Tx, block *btcutil.Block) {
// Nothing to do if nobody is listening for address notifications.
if len(m.addrNotifications) == 0 {
return
}
2014-02-24 15:10:59 +01:00
txHex := ""
wscNotified := make(map[chan bool]bool)
for i, txOut := range tx.MsgTx().TxOut {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
_, addrs, _, err := btcscript.ExtractPkScriptAddrs(
2014-02-24 15:10:59 +01:00
txOut.PkScript, m.server.server.btcnet)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
if err != nil {
continue
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, addr := range addrs {
encodedAddr := addr.EncodeAddress()
cmap, ok := m.addrNotifications[encodedAddr]
if !ok {
continue
}
2014-02-24 15:10:59 +01:00
if txHex == "" {
txHex = txHexString(tx)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
2014-02-24 15:10:59 +01:00
ntfn := btcws.NewRecvTxNtfn(txHex, blockDetails(block, tx.Index()))
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
marshalledJSON, err := json.Marshal(ntfn)
if err != nil {
rpcsLog.Errorf("Failed to marshal processedtx notification: %v", err)
2014-02-24 15:10:59 +01:00
continue
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
2014-02-24 15:10:59 +01:00
op := btcwire.NewOutPoint(tx.Sha(), uint32(i))
for wscQuit, wsc := range cmap {
m.addSpentRequest(wsc, op)
if !wscNotified[wscQuit] {
wscNotified[wscQuit] = true
wsc.QueueNotification(marshalledJSON)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
}
}
}
2014-02-24 15:10:59 +01:00
// NotifyForTx examines the inputs and outputs of the passed transaction,
// notifying websocket clients of outputs spending to a watched address
// and inputs spending a watched outpoint.
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
//
// This function is safe for concurrent access.
2014-02-24 15:10:59 +01:00
func (m *wsNotificationManager) NotifyForTx(tx *btcutil.Tx, block *btcutil.Block) {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
m.Lock()
defer m.Unlock()
2014-02-24 15:10:59 +01:00
m.notifyForTxIns(tx, block)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
m.notifyForTxOuts(tx, block)
}
2014-02-24 15:10:59 +01:00
// newRedeemingTxNotification returns a new marshalled redeemingtx notification
// with the passed parameters.
func newRedeemingTxNotification(txHex string, index int, block *btcutil.Block) ([]byte, error) {
// Create and marshal the notification.
ntfn := btcws.NewRedeemingTxNtfn(txHex, blockDetails(block, index))
return json.Marshal(ntfn)
}
2014-02-24 15:10:59 +01:00
// notifyForTxIns examines the inputs of the passed transaction and sends
// interested websocket clients a redeemingtx notification if any inputs
// spend a watched output. If block is non-nil, any matching spent
// requests are removed.
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
//
// This function MUST be called with the notification manager lock held.
2014-02-24 15:10:59 +01:00
func (m *wsNotificationManager) notifyForTxIns(tx *btcutil.Tx, block *btcutil.Block) {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Nothing to do if nobody is listening for spent notifications.
if len(m.spentNotifications) == 0 {
return
}
2014-02-24 15:10:59 +01:00
txHex := ""
wscNotified := make(map[chan bool]bool)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, txIn := range tx.MsgTx().TxIn {
prevOut := &txIn.PreviousOutpoint
if cmap, ok := m.spentNotifications[*prevOut]; ok {
2014-02-24 15:10:59 +01:00
if txHex == "" {
txHex = txHexString(tx)
}
marshalledJSON, err := newRedeemingTxNotification(txHex, tx.Index(), block)
if err != nil {
rpcsLog.Warnf("Failed to marshal redeemingtx notification: %v", err)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
continue
}
2014-02-24 15:10:59 +01:00
for wscQuit, wsc := range cmap {
if block != nil {
m.removeSpentRequest(wsc, prevOut)
}
if !wscNotified[wscQuit] {
wscNotified[wscQuit] = true
wsc.QueueNotification(marshalledJSON)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
}
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// NotifyBlockTXs examines the input and outputs of the passed transaction
// and sends websocket clients notifications they are interested in.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) NotifyBlockTXs(block *btcutil.Block) {
m.Lock()
defer m.Unlock()
// Nothing to do if there are no websocket clients registered to receive
// notifications about spent outpoints or payments to addresses.
if len(m.spentNotifications) == 0 && len(m.addrNotifications) == 0 {
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, tx := range block.Transactions() {
2014-02-24 15:10:59 +01:00
m.notifyForTxIns(tx, block)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
m.notifyForTxOuts(tx, block)
}
}
2014-02-24 15:10:59 +01:00
func blockDetails(block *btcutil.Block, txIndex int) *btcws.BlockDetails {
if block == nil {
return nil
}
blockSha, _ := block.Sha() // never errors
return &btcws.BlockDetails{
Height: int32(block.Height()),
Hash: blockSha.String(),
Index: txIndex,
Time: block.MsgBlock().Header.Timestamp.Unix(),
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// AddAddrRequest requests notifications to the passed websocket client when
// a transaction pays to the passed address.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) AddAddrRequest(wsc *wsClient, addr string) {
m.Lock()
defer m.Unlock()
// Track the request in the client as well so it can be quickly be
// removed on disconnect.
wsc.addrRequests[addr] = struct{}{}
// Add the client to the list to notify when the outpoint is seen.
// Create the list as needed.
cmap, ok := m.addrNotifications[addr]
if !ok {
cmap = make(map[chan bool]*wsClient)
m.addrNotifications[addr] = cmap
}
cmap[wsc.quit] = wsc
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// removeAddrRequest is the internal function which implements the public
// RemoveAddrRequest. See the comment for RemoveAddrRequest for more details.
//
// This function MUST be called with the notification manager lock held.
func (m *wsNotificationManager) removeAddrRequest(wsc *wsClient, addr string) {
// Remove the request tracking from the client.
delete(wsc.addrRequests, addr)
// Remove the client from the list to notify.
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
notifyMap, ok := m.addrNotifications[addr]
if !ok {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
rpcsLog.Warnf("Attempt to remove nonexistent addr request "+
"<%s> for websocket client %s", addr, wsc.addr)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
delete(notifyMap, wsc.quit)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Remove the map entry altogether if there are no more clients
// interested in it.
if len(notifyMap) == 0 {
delete(m.addrNotifications, addr)
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// RemoveAddrRequest removes a request from the passed websocket client to be
// notified when a transaction pays to the passed address.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) RemoveAddrRequest(wsc *wsClient, addr string) {
m.Lock()
defer m.Unlock()
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
m.removeAddrRequest(wsc, addr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// AddClient adds the passed websocket client to the notification manager.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) AddClient(wsc *wsClient) {
m.Lock()
defer m.Unlock()
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
m.clients[wsc.quit] = wsc
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// RemoveClient removes the passed websocket client and all notifications
// registered for it.
//
// This function is safe for concurrent access.
func (m *wsNotificationManager) RemoveClient(wsc *wsClient) {
m.Lock()
defer m.Unlock()
// Remove any requests made by the client as well as the client itself.
delete(m.blockNotifications, wsc.quit)
delete(m.txNotifications, wsc.quit)
for k := range wsc.spentRequests {
op := k
m.removeSpentRequest(wsc, &op)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for addr := range wsc.addrRequests {
m.removeAddrRequest(wsc, addr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
delete(m.clients, wsc.quit)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Shutdown disconnects all websocket clients the manager knows about.
func (m *wsNotificationManager) Shutdown() {
for _, wsc := range m.clients {
wsc.Disconnect()
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// newWsNotificationManager returns a new notification manager ready for use.
// See wsNotificationManager for more details.
func newWsNotificationManager(server *rpcServer) *wsNotificationManager {
return &wsNotificationManager{
server: server,
clients: make(map[chan bool]*wsClient),
blockNotifications: make(map[chan bool]*wsClient),
txNotifications: make(map[chan bool]*wsClient),
spentNotifications: make(map[btcwire.OutPoint]map[chan bool]*wsClient),
addrNotifications: make(map[string]map[chan bool]*wsClient),
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// wsResponse houses a message to send to the a connected websocket client as
// well as a channel to reply on when the message is sent.
type wsResponse struct {
msg []byte
doneChan chan bool
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// createMarshalledReply returns a new marshalled btcjson.Reply given the
// passed parameters. It will automatically convert errors that are not of
// the type *btcjson.Error to the appropriate type as needed.
func createMarshalledReply(id, result interface{}, replyErr error) ([]byte, error) {
var jsonErr *btcjson.Error
if replyErr != nil {
if jErr, ok := replyErr.(*btcjson.Error); !ok {
jsonErr = &btcjson.Error{
Code: btcjson.ErrInternal.Code,
Message: jErr.Error(),
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
response := btcjson.Reply{
Id: &id,
Result: result,
Error: jsonErr,
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
marshalledJSON, err := json.Marshal(response)
if err != nil {
return nil, err
}
return marshalledJSON, nil
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// wsClient provides an abstraction for handling a websocket client. The
// overall data flow is split into 3 main goroutines, a possible 4th goroutine
// for long-running operations (only started if request is made), and a
// websocket manager which is used to allow things such as broadcasting
// requested notifications to all connected websocket clients. Inbound
// messages are read via the inHandler goroutine and generally dispatched to
// their own handler. However, certain potentially long-running operations such
// as rescans, are sent to the asyncHander goroutine and are limited to one at a
// time. There are two outbound message types - one for responding to client
// requests and another for async notifications. Responses to client requests
// use SendMessage which employs a buffered channel thereby limiting the number
// of outstanding requests that can be made. Notifications are sent via
// QueueNotification which implements a queue via notificationQueueHandler to
// ensure sending notifications from other subsystems can't block. Ultimately,
// all messages are sent via the outHandler.
type wsClient struct {
sync.Mutex
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// server is the RPC server that is servicing the client.
server *rpcServer
// conn is the underlying websocket connection.
conn *websocket.Conn
// disconnected indicated whether or not the websocket client is
// disconnected.
disconnected bool
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// addr is the remote address of the client.
addr string
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// authenticated specifies whether a client has been authenticated
// and therefore is allowed to communicated over the websocket.
authenticated bool
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// verboseTxUpdates specifies whether a client has requested verbose
// information about all new transactions.
verboseTxUpdates bool
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// addrRequests is a set of addresses the caller has requested to be
// notified about. It is maintained here so all requests can be removed
// when a wallet disconnects. Owned by the notification manager.
addrRequests map[string]struct{}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// spentRequests is a set of unspent Outpoints a wallet has requested
// notifications for when they are spent by a processed transaction.
// Owned by the notification manager.
spentRequests map[btcwire.OutPoint]struct{}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Networking infrastructure.
asyncStarted bool
asyncChan chan btcjson.Cmd
ntfnChan chan []byte
sendChan chan wsResponse
quit chan bool
wg sync.WaitGroup
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleMessage is the main handler for incoming requests. It enforces
// authentication, parses the incoming json, looks up and executes handlers
// (including pass through for standard RPC commands), sends the appropriate
// response. It also detects commands which are marked as long-running and
// sends them off to the asyncHander for processing.
func (c *wsClient) handleMessage(msg string) {
if !c.authenticated {
// Disconnect immediately if the provided command fails to
// parse when the client is not already authenticated.
cmd, jsonErr := parseCmd([]byte(msg))
if jsonErr != nil {
c.Disconnect()
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Disconnect immediately if the first command is not
// authenticate when not already authenticated.
authCmd, ok := cmd.(*btcws.AuthenticateCmd)
if !ok {
rpcsLog.Warnf("Unauthenticated websocket message " +
"received")
c.Disconnect()
return
}
// Check credentials.
login := authCmd.Username + ":" + authCmd.Passphrase
auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(login))
authSha := sha256.Sum256([]byte(auth))
cmp := subtle.ConstantTimeCompare(authSha[:], c.server.authsha[:])
if cmp != 1 {
rpcsLog.Warnf("Auth failure.")
c.Disconnect()
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.authenticated = true
// Marshal and send response.
reply, err := createMarshalledReply(authCmd.Id(), nil, nil)
if err != nil {
rpcsLog.Errorf("Failed to marshal authenticate reply: "+
"%v", err.Error())
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.SendMessage(reply, nil)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Attmpt to parse the raw json into a known btcjson.Cmd.
cmd, jsonErr := parseCmd([]byte(msg))
if jsonErr != nil {
// Use the provided id for errors when a valid JSON-RPC message
// was parsed. Requests with no IDs are ignored.
var id interface{}
if cmd != nil {
id = cmd.Id()
if id == nil {
return
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Marshal and send response.
reply, err := createMarshalledReply(id, nil, jsonErr)
if err != nil {
rpcsLog.Errorf("Failed to marshal parse failure "+
"reply: %v", err)
return
}
c.SendMessage(reply, nil)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
rpcsLog.Debugf("Received command <%s> from %s", cmd.Method(), c.addr)
// Disconnect if already authenticated and another authenticate command
// is received.
if _, ok := cmd.(*btcws.AuthenticateCmd); ok {
rpcsLog.Warnf("Websocket client %s is already authenticated",
c.addr)
c.Disconnect()
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// When the command is marked as a long-running command, send it off
// to the asyncHander goroutine for processing.
if _, ok := wsAsyncHandlers[cmd.Method()]; ok {
// Start up the async goroutine for handling long-running
// requests asynchonrously if needed.
if !c.asyncStarted {
rpcsLog.Tracef("Starting async handler for %s", c.addr)
c.wg.Add(1)
go c.asyncHandler()
c.asyncStarted = true
}
c.asyncChan <- cmd
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Lookup the websocket extension for the command and if it doesn't
// exist fallback to handling the command as a standard command.
wsHandler, ok := wsHandlers[cmd.Method()]
if !ok {
// No websocket-specific handler so handle like a legacy
// RPC connection.
response := standardCmdReply(cmd, c.server)
reply, err := json.Marshal(response)
if err != nil {
rpcsLog.Errorf("Failed to marshal reply for <%s> "+
"command: %v", cmd.Method(), err)
return
}
c.SendMessage(reply, nil)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Invoke the handler and marshal and send response.
result, jsonErr := wsHandler(c, cmd)
reply, err := createMarshalledReply(cmd.Id(), result, jsonErr)
if err != nil {
rpcsLog.Errorf("Failed to marshal reply for <%s> command: %v",
cmd.Method(), err)
return
}
c.SendMessage(reply, nil)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// inHandler handles all incoming messages for the websocket connection. It
// must be run as a goroutine.
func (c *wsClient) inHandler() {
out:
for {
// Break out of the loop once the quit channel has been closed.
// Use a non-blocking select here so we fall through otherwise.
select {
case <-c.quit:
break out
default:
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
var msg string
if err := websocket.Message.Receive(c.conn, &msg); err != nil {
// Log the error if it's not due to disconnecting.
if err != io.EOF {
rpcsLog.Errorf("Websocket receive error from "+
"%s: %v", c.addr, err)
}
break out
}
c.handleMessage(msg)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Ensure the connection is closed.
c.Disconnect()
c.wg.Done()
rpcsLog.Tracef("Websocket client input handler done for %s", c.addr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// notificationQueueHandler handles the queueing of outgoing notifications for
// the websocket client. This runs as a muxer for various sources of input to
// ensure that queueing up notifications to be sent will not block. Otherwise,
// slow clients could bog down the other systems (such as the mempool or block
// manager) which are queueing the data. The data is passed on to outHandler to
// actually be written. It must be run as a goroutine.
func (c *wsClient) notificationQueueHandler() {
ntfnSentChan := make(chan bool, 1) // nonblocking sync
// pendingNtfns is used as a queue for notifications that are ready to
// be sent once there are no outstanding notifications currently being
// sent. The waiting flag is used over simply checking for items in the
// pending list to ensure cleanup knows what has and hasn't been sent
// to the outHandler. Currently no special cleanup is needed, however
// if something like a done channel is added to notifications in the
// future, not knowing what has and hasn't been sent to the outHandler
// (and thus who should respond to the done channel) would be
// problematic without using this approach.
pendingNtfns := list.New()
waiting := false
out:
for {
select {
// This channel is notified when a message is being queued to
// be sent across the network socket. It will either send the
// message immediately if a send is not already in progress, or
// queue the message to be sent once the other pending messages
// are sent.
case msg := <-c.ntfnChan:
if !waiting {
c.SendMessage(msg, ntfnSentChan)
} else {
pendingNtfns.PushBack(msg)
}
waiting = true
// This channel is notified when a notification has been sent
// across the network socket.
case <-ntfnSentChan:
// No longer waiting if there are no more messages in
// the pending messages queue.
next := pendingNtfns.Front()
if next == nil {
waiting = false
continue
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Notify the outHandler about the next item to
// asynchronously send.
msg := pendingNtfns.Remove(next).([]byte)
c.SendMessage(msg, ntfnSentChan)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
case <-c.quit:
break out
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Drain any wait channels before exiting so nothing is left waiting
// around to send.
cleanup:
for {
select {
case <-c.ntfnChan:
case <-ntfnSentChan:
default:
break cleanup
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
c.wg.Done()
rpcsLog.Tracef("Websocket client notification queue handler done "+
"for %s", c.addr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// outHandler handles all outgoing messages for the websocket connection. It
// must be run as a goroutine. It uses a buffered channel to serialize output
// messages while allowing the sender to continue running asynchronously. It
// must be run as a goroutine.
func (c *wsClient) outHandler() {
out:
for {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Send any messages ready for send until the quit channel is
// closed.
select {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
case r := <-c.sendChan:
err := websocket.Message.Send(c.conn, string(r.msg))
if err != nil {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.Disconnect()
break out
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
if r.doneChan != nil {
r.doneChan <- true
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
case <-c.quit:
break out
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Drain any wait channels before exiting so nothing is left waiting
// around to send.
cleanup:
for {
select {
case r := <-c.sendChan:
if r.doneChan != nil {
r.doneChan <- false
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
default:
break cleanup
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.wg.Done()
rpcsLog.Tracef("Websocket client output handler done for %s", c.addr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// asyncHandler handles all long-running requests such as rescans which are
// not run directly in the inHandler routine unlike most requests. This allows
// normal quick requests to continue to be processed and responded to even while
// lengthy operations are underway. Only one long-running operation is
// permitted at a time, so multiple long-running requests are queued and
// serialized. It must be run as a goroutine. Also, this goroutine is not
// started until/if the first long-running request is made.
func (c *wsClient) asyncHandler() {
asyncHandlerDoneChan := make(chan bool, 1) // nonblocking sync
pendingCmds := list.New()
waiting := false
// runHandler runs the handler for the passed command and sends the
// reply.
runHandler := func(cmd btcjson.Cmd) {
wsHandler, ok := wsHandlers[cmd.Method()]
if !ok {
rpcsLog.Warnf("No handler for command <%s>",
cmd.Method())
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Invoke the handler and marshal and send response.
result, jsonErr := wsHandler(c, cmd)
reply, err := createMarshalledReply(cmd.Id(), result, jsonErr)
if err != nil {
rpcsLog.Errorf("Failed to marshal reply for <%s> "+
"command: %v", cmd.Method(), err)
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.SendMessage(reply, nil)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
out:
for {
select {
case cmd := <-c.asyncChan:
if !waiting {
c.wg.Add(1)
go func(cmd btcjson.Cmd) {
runHandler(cmd)
asyncHandlerDoneChan <- true
c.wg.Done()
}(cmd)
} else {
pendingCmds.PushBack(cmd)
}
waiting = true
case <-asyncHandlerDoneChan:
// No longer waiting if there are no more messages in
// the pending messages queue.
next := pendingCmds.Front()
if next == nil {
waiting = false
continue
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Notify the outHandler about the next item to
// asynchronously send.
element := pendingCmds.Remove(next)
c.wg.Add(1)
go func(cmd btcjson.Cmd) {
runHandler(cmd)
asyncHandlerDoneChan <- true
c.wg.Done()
}(element.(btcjson.Cmd))
case <-c.quit:
break out
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Drain any wait channels before exiting so nothing is left waiting
// around to send.
cleanup:
for {
select {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
case <-c.asyncChan:
case <-asyncHandlerDoneChan:
default:
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
break cleanup
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.wg.Done()
rpcsLog.Tracef("Websocket client async handler done for %s", c.addr)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// SendMessage sends the passed json to the websocket client. It is backed
// by a buffered channel, so it will not block until the send channel is full.
// Note however that QueueNotification must be used for sending async
// notifications instead of the this function. This approach allows a limit to
// the number of outstanding requests a client can make without preventing or
// blocking on async notifications.
func (c *wsClient) SendMessage(marshalledJSON []byte, doneChan chan bool) {
// Don't send the message if disconnected.
if c.Disconnected() {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
if doneChan != nil {
doneChan <- false
}
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.sendChan <- wsResponse{msg: marshalledJSON, doneChan: doneChan}
}
2014-02-24 15:10:59 +01:00
// ErrClientQuit describes the error where a client send is not processed due
// to the client having already been disconnected or dropped.
var ErrClientQuit = errors.New("client quit")
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// QueueMessage queues the passed notification to be sent to the websocket
// client. This function, as the name implies, is only intended for
// notifications since it has additional logic to prevent other subsystems, such
// as the memory pool and block manager, from blocking even when the send
// channel is full.
2014-02-24 15:10:59 +01:00
//
// If the client is in the process of shutting down, this function returns
// ErrClientQuit. This is intended to be checked by long-running notification
// handlers to stop processing if there is no more work needed to be done.
func (c *wsClient) QueueNotification(marshalledJSON []byte) error {
// Don't queue the message if disconnected.
if c.Disconnected() {
2014-02-24 15:10:59 +01:00
return ErrClientQuit
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
c.ntfnChan <- marshalledJSON
2014-02-24 15:10:59 +01:00
return nil
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
// Disconnected returns whether or not the websocket client is disconnected.
func (c *wsClient) Disconnected() bool {
c.Lock()
defer c.Unlock()
return c.disconnected
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Disconnect disconnects the websocket client.
func (c *wsClient) Disconnect() {
c.Lock()
defer c.Unlock()
// Nothing to do if already disconnected.
if c.disconnected {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
return
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
rpcsLog.Tracef("Disconnecting websocket client %s", c.addr)
close(c.quit)
c.conn.Close()
c.disconnected = true
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Start begins processing input and output messages.
func (c *wsClient) Start() {
rpcsLog.Tracef("Starting websocket client %s", c.addr)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Start processing input and output.
c.wg.Add(3)
go c.inHandler()
go c.notificationQueueHandler()
go c.outHandler()
}
// WaitForShutdown blocks until the websocket client goroutines are stopped
// and the connection is closed.
func (c *wsClient) WaitForShutdown() {
c.wg.Wait()
}
// newWebsocketClient returns a new websocket client given the notification
// manager, websocket connection, remote address, and whether or not the client
// has already been authenticated (via HTTP Basic access authentication). The
// returned client is ready to start. Once started, the client will process
// incoming and outgoing messages in separate goroutines complete with queueing
// and asynchrous handling for long-running operations.
func newWebsocketClient(server *rpcServer, conn *websocket.Conn,
remoteAddr string, authenticated bool) *wsClient {
return &wsClient{
conn: conn,
addr: remoteAddr,
authenticated: authenticated,
server: server,
addrRequests: make(map[string]struct{}),
spentRequests: make(map[btcwire.OutPoint]struct{}),
ntfnChan: make(chan []byte, 1), // nonblocking sync
asyncChan: make(chan btcjson.Cmd, 1), // nonblocking sync
sendChan: make(chan wsResponse, websocketSendBufferSize),
quit: make(chan bool),
}
}
// handleGetBestBlock implements the getbestblock command extension
// for websocket connections.
func handleGetBestBlock(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
// All other "get block" commands give either the height, the
// hash, or both but require the block SHA. This gets both for
// the best block.
sha, height, err := wsc.server.server.db.NewestSha()
if err != nil {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
return nil, &btcjson.ErrBestBlockHash
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// TODO(jrick): need a btcws type for the result.
result := map[string]interface{}{
"hash": sha.String(),
"height": height,
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
return result, nil
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleGetCurrentNet implements the getcurrentnet command extension
// for websocket connections.
func handleGetCurrentNet(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
return wsc.server.server.btcnet, nil
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleNotifyBlocks implements the notifyblocks command extension for
// websocket connections.
func handleNotifyBlocks(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
wsc.server.ntfnMgr.AddBlockUpdateRequest(wsc)
return nil, nil
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleNotifySpent implements the notifyspent command extension for
// websocket connections.
func handleNotifySpent(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
cmd, ok := icmd.(*btcws.NotifySpentCmd)
if !ok {
return nil, &btcjson.ErrInternal
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
wsc.server.ntfnMgr.AddSpentRequest(wsc, cmd.OutPoint)
return nil, nil
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleNotifyAllNewTXs implements the notifyallnewtxs command extension for
// websocket connections.
func handleNotifyAllNewTXs(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
cmd, ok := icmd.(*btcws.NotifyAllNewTXsCmd)
if !ok {
return nil, &btcjson.ErrInternal
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
wsc.verboseTxUpdates = cmd.Verbose
wsc.server.ntfnMgr.AddNewTxRequest(wsc)
return nil, nil
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleNotifyNewTXs implements the notifynewtxs command extension for
// websocket connections.
func handleNotifyNewTXs(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
cmd, ok := icmd.(*btcws.NotifyNewTXsCmd)
if !ok {
return nil, &btcjson.ErrInternal
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, addrStr := range cmd.Addresses {
addr, err := btcutil.DecodeAddr(addrStr)
if err != nil {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
e := btcjson.Error{
Code: btcjson.ErrInvalidAddressOrKey.Code,
Message: fmt.Sprintf("Invalid address or key: %v", addrStr),
}
return nil, &e
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
wsc.server.ntfnMgr.AddAddrRequest(wsc, addr.EncodeAddress())
}
return nil, nil
}
// rescanBlock rescans all transactions in a single block. This is a helper
// function for handleRescan.
2014-02-24 15:10:59 +01:00
func rescanBlock(wsc *wsClient, cmd *btcws.RescanCmd, blk *btcutil.Block,
unspent map[btcwire.OutPoint]struct{}) {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for _, tx := range blk.Transactions() {
2014-02-24 15:10:59 +01:00
// Hexadecimal representation of this tx. Only created if
// needed, and reused for later notifications if already made.
var txHex string
// All inputs and outputs must be iterated through to correctly
// modify the unspent map, however, just a single notification
// for any matching transaction inputs or outputs should be
// created and sent.
spentNotified := false
recvNotified := false
for _, txin := range tx.MsgTx().TxIn {
if _, ok := unspent[txin.PreviousOutpoint]; ok {
delete(unspent, txin.PreviousOutpoint)
if spentNotified {
continue
}
if txHex == "" {
txHex = txHexString(tx)
}
marshalledJSON, err := newRedeemingTxNotification(txHex, tx.Index(), blk)
if err != nil {
rpcsLog.Errorf("Failed to marshal redeemingtx notification: %v", err)
continue
}
err = wsc.QueueNotification(marshalledJSON)
// Stop the rescan early if the websocket client
// disconnected.
if err == ErrClientQuit {
return
}
spentNotified = true
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
for txOutIdx, txout := range tx.MsgTx().TxOut {
2014-02-24 15:10:59 +01:00
_, addrs, _, _ := btcscript.ExtractPkScriptAddrs(
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
txout.PkScript, wsc.server.server.btcnet)
for _, addr := range addrs {
encodedAddr := addr.EncodeAddress()
if _, ok := cmd.Addresses[encodedAddr]; !ok {
continue
}
2014-02-24 15:10:59 +01:00
unspent[*btcwire.NewOutPoint(tx.Sha(), uint32(txOutIdx))] = struct{}{}
if recvNotified {
continue
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
2014-02-24 15:10:59 +01:00
if txHex == "" {
txHex = txHexString(tx)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
}
2014-02-24 15:10:59 +01:00
ntfn := btcws.NewRecvTxNtfn(txHex, blockDetails(blk, tx.Index()))
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
marshalledJSON, err := json.Marshal(ntfn)
if err != nil {
2014-02-24 15:10:59 +01:00
rpcsLog.Errorf("Failed to marshal recvtx notification: %v", err)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
return
}
2014-02-24 15:10:59 +01:00
err = wsc.QueueNotification(marshalledJSON)
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// Stop the rescan early if the websocket client
// disconnected.
2014-02-24 15:10:59 +01:00
if err == ErrClientQuit {
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
return
}
2014-02-24 15:10:59 +01:00
recvNotified = true
}
}
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// handleRescan implements the rescan command extension for websocket
// connections.
func handleRescan(wsc *wsClient, icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
cmd, ok := icmd.(*btcws.RescanCmd)
if !ok {
return nil, &btcjson.ErrInternal
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
numAddrs := len(cmd.Addresses)
if numAddrs == 1 {
rpcsLog.Info("Beginning rescan for 1 address")
} else {
rpcsLog.Infof("Beginning rescan for %d addresses", numAddrs)
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
minBlock := int64(cmd.BeginBlock)
maxBlock := int64(cmd.EndBlock)
2014-02-24 15:10:59 +01:00
unspent := make(map[btcwire.OutPoint]struct{})
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
// FetchHeightRange may not return a complete list of block shas for
// the given range, so fetch range as many times as necessary.
db := wsc.server.server.db
for {
hashList, err := db.FetchHeightRange(minBlock, maxBlock)
if err != nil {
rpcsLog.Errorf("Error looking up block range: %v", err)
return nil, &btcjson.ErrDatabase
}
if len(hashList) == 0 {
break
}
for i := range hashList {
blk, err := db.FetchBlockBySha(&hashList[i])
if err != nil {
rpcsLog.Errorf("Error looking up block sha: %v", err)
return nil, &btcjson.ErrDatabase
}
// A select statement is used to stop rescans if the
// client requesting the rescan has disconnected.
select {
case <-wsc.quit:
rpcsLog.Debugf("Stopped rescan at height %v for disconnected client",
blk.Height())
return nil, nil
default:
2014-02-24 15:10:59 +01:00
rescanBlock(wsc, cmd, blk, unspent)
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
if maxBlock-minBlock > int64(len(hashList)) {
minBlock += int64(len(hashList))
} else {
break
}
}
Rework and improve websocket notification system. This commit refactors the entire websocket client code to resolve several issues with the previous implementation. Note that this commit does not change the public API for websockets. It only consists of internal improvements. The following is the major issues which have been addressed: - A slow websocket client could impede notifications to all clients - Long-running operations such as rescans would block all other requests until it had completed - The above two points taken together could lead to apparant hangs since the client doing the rescan would eventually run out of channel buffer and block the entire group of clients until the rescan completed - Disconnecting a websocket during certain operations could lead to a hang - Stopping the rpc server with operations under way could lead to a hang - There were no limits to the number of websocket clients that could connect The following is a summary of the major changes: - The websocket code has been split into two entities: a connection/notification manager and a websocket client - The new connection/notification manager acts as the entry point from the rest of the subsystems to feed data which potentially needs to notify clients - Each websocket client now has its own instance of the new websocket client type which controls its own lifecycle - The data flow has been completely redesigned to closely resemble the peer data flow - Each websocket now has its own long-lived goroutines for input, output, and queuing of notifications - Notifications use the new notification queue goroutine along with queueing to ensure they dont't block on stalled or slow peers - There is a new infrastructure for asynchronously executing long-running commands such as a rescan while still allowing the faster operations to continue to be serviced by the same client - Since long-running operations now run asynchronously, they have been limited to one at a time - Added a limit of 10 websocket clients. This is hard coded for now, but will be made configurable in the future Taken together these changes make the code far easier to reason about and update as well solve the aforementioned issues. Further optimizations to improve performance are possible in regards to the way the connection/notification manager works, however this commit already contains a ton of changes, so they are being left for another time.
2014-02-19 00:23:33 +01:00
rpcsLog.Info("Finished rescan")
return nil, nil
}