mirrors/vector-im-hydrogen-web
1
0
Fork 0
mirror of https://github.com/vector-im/hydrogen-web.git synced 2024-12-22 19:14:52 +01:00
Code Issues Projects Releases Wiki Activity

Merge pull request #705 from vector-im/bwindels/calls

Browse Source 
Group calls
This commit is contained in:
Bruno Windels 2023-02-10 11:36:39 +01:00 committed by GitHub
commit 8cccc1dfaf
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
142 changed files with 8147 additions and 2349 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.eslintrc.js
View File

@ -17,6 +17,7 @@ module.exports = {
"globals": {
"DEFINE_VERSION": "readonly",
"DEFINE_GLOBAL_HASH": "readonly",
"DEFINE_PROJECT_DIR": "readonly",
// only available in sw.js
"DEFINE_UNHASHED_PRECACHED_ASSETS": "readonly",
"DEFINE_HASHED_PRECACHED_ASSETS": "readonly",

15
doc/error-handling.md Normal file
View File

@ -0,0 +1,15 @@
# Error handling
Ideally, every error that is unexpected and can't be automatically recovered from without degrading the experience is shown in the UI. This is the task of the view model, and you can use `ErrorReportViewModel` for this purpose, a dedicated base view model class. It exposes a child view model, `ErrorViewModel`, when `reportError` is called which can be paired with `ErrorView` in the view to present an error message from which debug logs can also be sent.
Methods on classes from the `matrix` layer can often throw errors and those errors should be caught in the view model and reported to the UI. When inheriting from `ErrorReportViewModel`, there is the low-level `reportError` method, but typically you'd use the convenience method `logAndCatch`. The latter makes it easy to get both error handlikng and logging right. You would typically use `logAndCatch` for every public method in the view model (e.g methods called from the view or from the parent view model). It calls a callback within a log item and also a try catch that reports the error.
## Sync errors & ErrorBoundary
There are some errors that are thrown during background processes though, most notably the sync loop. These processes are not triggered by the view model directly, and hence there is not always a method call they can wrap in a try/catch. For this, there is the `ErrorBoundary` utility class. Since almost all aspects of the client can be updated through the sync loop, it is also not too helpful if there is only one try/catch around the whole sync and we stop sync if something goes wrong.
Instead, it's more helpful to split up the error handling into different scopes, where errors are stored and not rethrown when leaving the scope. One example is to have a scope per room. In this way, we can isolate an error occuring during sync to a specific room, and report it in the UI of that room. This is typically where you would use `reportError` from `ErrorReportViewModel` rather than `logAndCatch`. You observe changes from your model in the view model (see docs on updates), and if the `error` property is set (by the `ErrorBoundary`), you call reportError with it. You can do this repeatedly without problems, if the same error is already reported, it's a No-Op.
### `writeSync` and preventing data loss when dealing with errors.
There is an extra complication though. The `writeSync` sync lifecycle step should not swallow any errors, or data loss can occur. This is because the whole `writeSync` lifecycle step writes all changes (for all rooms, the session, ...) for a sync response in one transaction (including the sync token), and aborts the transaction and stops sync if there is an error thrown during this step. So if there is an error in `writeSync` of a given room, it's fair to assume not all changes it was planning to write were passed to the transaction, as it got interrupted by the exception. Therefore, if we would swallow the error, data loss can occur as we'd not get another chance to write these changes to disk as we would have advanced the sync token. Therefore, code in the `writeSync` lifecycle step should be written defensively but always throw.

2
doc/implementation planning/RELATIONS.md
View File

@ -237,7 +237,7 @@ room.sendEvent(eventEntry.eventType, replacement);
## Replies
```js
const reply = eventEntry.reply({});
const reply = eventEntry.createReplyContent({});
room.sendEvent("m.room.message", reply);
```

1
package.json
View File

@ -32,6 +32,7 @@
},
"homepage": "https://github.com/vector-im/hydrogen-web/#readme",
"devDependencies": {
"@matrixdotorg/structured-logviewer": "^0.0.3",
"@playwright/test": "^1.27.1",
"@typescript-eslint/eslint-plugin": "^4.29.2",
"@typescript-eslint/parser": "^4.29.2",

51
scripts/logviewer/file.js
View File

@ -1,51 +0,0 @@
/*
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
export function openFile(mimeType = null) {
const input = document.createElement("input");
input.setAttribute("type", "file");
input.className = "hidden";
if (mimeType) {
input.setAttribute("accept", mimeType);
}
const promise = new Promise((resolve, reject) => {
const checkFile = () => {
input.removeEventListener("change", checkFile, true);
const file = input.files[0];
document.body.removeChild(input);
if (file) {
resolve(file);
} else {
reject(new Error("no file picked"));
}
}
input.addEventListener("change", checkFile, true);
});
// IE11 needs the input to be attached to the document
document.body.appendChild(input);
input.click();
return promise;
}
export function readFileAsText(file) {
const reader = new FileReader();
const promise = new Promise((resolve, reject) => {
reader.addEventListener("load", evt => resolve(evt.target.result));
reader.addEventListener("error", evt => reject(evt.target.error));
});
reader.readAsText(file);
return promise;
}

110
scripts/logviewer/html.js
View File

@ -1,110 +0,0 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// DOM helper functions
export function isChildren(children) {
// children should be an not-object (that's the attributes), or a domnode, or an array
return typeof children !== "object" || !!children.nodeType || Array.isArray(children);
}
export function classNames(obj, value) {
return Object.entries(obj).reduce((cn, [name, enabled]) => {
if (typeof enabled === "function") {
enabled = enabled(value);
}
if (enabled) {
return cn + (cn.length ? " " : "") + name;
} else {
return cn;
}
}, "");
}
export function setAttribute(el, name, value) {
if (name === "className") {
name = "class";
}
if (value === false) {
el.removeAttribute(name);
} else {
if (value === true) {
value = name;
}
el.setAttribute(name, value);
}
}
export function el(elementName, attributes, children) {
return elNS(HTML_NS, elementName, attributes, children);
}
export function elNS(ns, elementName, attributes, children) {
if (attributes && isChildren(attributes)) {
children = attributes;
attributes = null;
}
const e = document.createElementNS(ns, elementName);
if (attributes) {
for (let [name, value] of Object.entries(attributes)) {
if (name === "className" && typeof value === "object" && value !== null) {
value = classNames(value);
}
setAttribute(e, name, value);
}
}
if (children) {
if (!Array.isArray(children)) {
children = [children];
}
for (let c of children) {
if (!c.nodeType) {
c = text(c);
}
e.appendChild(c);
}
}
return e;
}
export function text(str) {
return document.createTextNode(str);
}
export const HTML_NS = "http://www.w3.org/1999/xhtml";
export const SVG_NS = "http://www.w3.org/2000/svg";
export const TAG_NAMES = {
[HTML_NS]: [
"br", "a", "ol", "ul", "li", "div", "h1", "h2", "h3", "h4", "h5", "h6",
"p", "strong", "em", "span", "img", "section", "main", "article", "aside",
"pre", "button", "time", "input", "textarea", "label", "form", "progress", "output"],
[SVG_NS]: ["svg", "circle"]
};
export const tag = {};
for (const [ns, tags] of Object.entries(TAG_NAMES)) {
for (const tagName of tags) {
tag[tagName] = function(attributes, children) {
return elNS(ns, tagName, attributes, children);
}
}
}

209
scripts/logviewer/index.html
View File

@ -1,209 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<style type="text/css">
html, body {
height: 100%;
}
body {
font-family: sans-serif;
font-size: 1rem;
margin: 0;
display: grid;
grid-template-areas: "nav nav" "items details";
grid-template-columns: 1fr 400px;
grid-template-rows: auto 1fr;
min-height: 0;
}
main {
grid-area: items;
min-width: 0;
min-height: 0;
overflow-y: auto;
padding: 8px;
}
main section h2 {
margin: 2px 14px;
font-size: 1rem;
}
aside {
grid-area: details;
padding: 8px;
}
aside h3 {
word-wrap: anywhere;
}
aside p {
margin: 2px 0;
}
aside .values li span {
word-wrap: ;
word-wrap: anywhere;
padding: 4px;
}
aside .values {
list-style: none;
padding: 0;
border: 1px solid lightgray;
}
aside .values span.key {
width: 30%;
display: block;
}
aside .values span.value {
width: 70%;
display: block;
white-space: pre-wrap;
}
aside .values li {
display: flex;
}
aside .values li:not(:first-child) {
border-top: 1px solid lightgray;
}
nav {
grid-area: nav;
}
.timeline li:not(.expanded) > ol {
display: none;
}
.timeline li > div {
display: flex;
}
.timeline .toggleExpanded {
border: none;
background: none;
width: 24px;
height: 24px;
margin-right: 4px;
cursor: pointer;
}
.timeline .toggleExpanded:before {
content: "▶";
}
.timeline li.expanded > div > .toggleExpanded:before {
content: "▼";
}
.timeline ol {
list-style: none;
padding: 0 0 0 20px;
margin: 0;
}
.timeline .item {
--hue: 100deg;
--brightness: 80%;
background-color: hsl(var(--hue), 60%, var(--brightness));
border: 1px solid hsl(var(--hue), 60%, calc(var(--brightness) - 40%));
border-radius: 4px;
padding: 2px;
display: flex;
margin: 1px;
flex: 1;
min-width: 0;
color: inherit;
text-decoration: none;
}
.timeline .item:not(.has-children) {
margin-left: calc(24px + 4px + 1px);
}
.timeline .item .caption {
white-space: nowrap;
text-overflow: ellipsis;
overflow: hidden;
flex: 1;
}
.timeline .item.level-3 {
--brightness: 90%;
}
.timeline .item.level-2 {
--brightness: 95%;
}
.timeline .item.level-5 {
--brightness: 80%;
}
.timeline .item.level-6, .timeline .item.level-7 {
--hue: 0deg !important;
}
.timeline .item.level-7 {
--brightness: 50%;
color: white;
}
.timeline .item.type-network {
--hue: 30deg;
}
.timeline .item.type-navigation {
--hue: 200deg;
}
.timeline .item.selected {
background-color: Highlight;
border-color: Highlight;
color: HighlightText;
}
.timeline .item.highlighted {
background-color: fuchsia;
color: white;
}
.hidden {
display: none;
}
#highlight {
width: 300px;
}
nav form {
display: inline;
}
</style>
</head>
<body>
<nav>
<button id="openFile">Open log file</button>
<button id="collapseAll">Collapse all</button>
<button id="hideCollapsed">Hide collapsed root items</button>
<button id="hideHighlightedSiblings" title="Hide collapsed siblings of highlighted">Hide non-highlighted</button>
<button id="showAll">Show all</button>
<form id="highlightForm">
<input type="text" id="highlight" name="highlight" placeholder="Highlight a search term" autocomplete="on">
<output id="highlightMatches"></output>
</form>
</nav>
<main></main>
<aside></aside>
<script type="module" src="main.js"></script>
</body>
</html>

398
scripts/logviewer/main.js
View File

@ -1,398 +0,0 @@
/*
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {tag as t} from "./html.js";
import {openFile, readFileAsText} from "./file.js";
const main = document.querySelector("main");
let selectedItemNode;
let rootItem;
let itemByRef;
const logLevels = [undefined, "All", "Debug", "Detail", "Info", "Warn", "Error", "Fatal", "Off"];
main.addEventListener("click", event => {
if (event.target.classList.contains("toggleExpanded")) {
const li = event.target.parentElement.parentElement;
li.classList.toggle("expanded");
} else {
// allow clicking any links other than .item in the timeline, like refs
if (event.target.tagName === "A" && !event.target.classList.contains("item")) {
return;
}
const itemNode = event.target.closest(".item");
if (itemNode) {
// we don't want scroll to jump when clicking
// so prevent default behaviour, and select and push to history manually
event.preventDefault();
selectNode(itemNode);
history.pushState(null, null, `#${itemNode.id}`);
}
}
});
window.addEventListener("hashchange", () => {
const id = window.location.hash.substr(1);
const itemNode = document.getElementById(id);
if (itemNode && itemNode.closest("main")) {
selectNode(itemNode);
itemNode.scrollIntoView({behavior: "smooth", block: "nearest"});
}
});
function selectNode(itemNode) {
if (selectedItemNode) {
selectedItemNode.classList.remove("selected");
}
selectedItemNode = itemNode;
selectedItemNode.classList.add("selected");
let item = rootItem;
let parent;
const indices = selectedItemNode.id.split("/").map(i => parseInt(i, 10));
for(const i of indices) {
parent = item;
item = itemChildren(item)[i];
}
showItemDetails(item, parent, selectedItemNode);
}
function stringifyItemValue(value) {
if (typeof value === "object" && value !== null) {
return JSON.stringify(value, undefined, 2);
} else {
return value + "";
}
}
function showItemDetails(item, parent, itemNode) {
const parentOffset = itemStart(parent) ? `${itemStart(item) - itemStart(parent)}ms` : "none";
const expandButton = t.button("Expand recursively");
expandButton.addEventListener("click", () => expandResursively(itemNode.parentElement.parentElement));
const start = itemStart(item);
const aside = t.aside([
t.h3(itemCaption(item)),
t.p([t.strong("Log level: "), logLevels[itemLevel(item)]]),
t.p([t.strong("Error: "), itemError(item) ? `${itemError(item).name} ${itemError(item).stack}` : "none"]),
t.p([t.strong("Parent offset: "), parentOffset]),
t.p([t.strong("Start: "), new Date(start).toString(), ` (${start})`]),
t.p([t.strong("Duration: "), `${itemDuration(item)}ms`]),
t.p([t.strong("Child count: "), itemChildren(item) ? `${itemChildren(item).length}` : "none"]),
t.p([t.strong("Forced finish: "), (itemForcedFinish(item) || false) + ""]),
t.p(t.strong("Values:")),
t.ul({class: "values"}, Object.entries(itemValues(item)).map(([key, value]) => {
let valueNode;
if (key === "ref") {
const refItem = itemByRef.get(value);
if (refItem) {
valueNode = t.a({href: `#${refItem.id}`}, itemCaption(refItem));
} else {
valueNode = `unknown ref ${value}`;
}
} else {
valueNode = stringifyItemValue(value);
}
return t.li([
t.span({className: "key"}, normalizeValueKey(key)),
t.span({className: "value"}, valueNode)
]);
})),
t.p(expandButton)
]);
document.querySelector("aside").replaceWith(aside);
}
function expandResursively(li) {
li.classList.add("expanded");
const ol = li.querySelector("ol");
if (ol) {
const len = ol.children.length;
for (let i = 0; i < len; i += 1) {
expandResursively(ol.children[i]);
}
}
}
document.getElementById("openFile").addEventListener("click", loadFile);
function getRootItemHeader(prevItem, item) {
if (prevItem) {
const diff = itemStart(item) - itemEnd(prevItem);
if (diff >= 0) {
return `+ ${formatTime(diff)}`;
} else {
const overlap = -diff;
if (overlap >= itemDuration(item)) {
return `ran entirely in parallel with`;
} else {
return `ran ${formatTime(-diff)} in parallel with`;
}
}
} else {
return new Date(itemStart(item)).toString();
}
}
async function loadFile() {
const file = await openFile();
const json = await readFileAsText(file);
const logs = JSON.parse(json);
logs.items.sort((a, b) => itemStart(a) - itemStart(b));
rootItem = {c: logs.items};
itemByRef = new Map();
preprocessRecursively(rootItem, null, itemByRef, []);
const fragment = logs.items.reduce((fragment, item, i, items) => {
const prevItem = i === 0 ? null : items[i - 1];
fragment.appendChild(t.section([
t.h2(getRootItemHeader(prevItem, item)),
t.div({className: "timeline"}, t.ol(itemToNode(item, [i])))
]));
return fragment;
}, document.createDocumentFragment());
main.replaceChildren(fragment);
}
// TODO: make this use processRecursively
function preprocessRecursively(item, parentElement, refsMap, path) {
item.s = (parentElement?.s || 0) + item.s;
if (itemRefSource(item)) {
refsMap.set(itemRefSource(item), item);
}
if (itemChildren(item)) {
for (let i = 0; i < itemChildren(item).length; i += 1) {
// do it in advance for a child as we don't want to do it for the rootItem
const child = itemChildren(item)[i];
const childPath = path.concat(i);
child.id = childPath.join("/");
preprocessRecursively(child, item, refsMap, childPath);
}
}
}
const MS_IN_SEC = 1000;
const MS_IN_MIN = MS_IN_SEC * 60;
const MS_IN_HOUR = MS_IN_MIN * 60;
const MS_IN_DAY = MS_IN_HOUR * 24;
function formatTime(ms) {
let str = "";
if (ms > MS_IN_DAY) {
const days = Math.floor(ms / MS_IN_DAY);
ms -= days * MS_IN_DAY;
str += `${days}d`;
}
if (ms > MS_IN_HOUR) {
const hours = Math.floor(ms / MS_IN_HOUR);
ms -= hours * MS_IN_HOUR;
str += `${hours}h`;
}
if (ms > MS_IN_MIN) {
const mins = Math.floor(ms / MS_IN_MIN);
ms -= mins * MS_IN_MIN;
str += `${mins}m`;
}
if (ms > MS_IN_SEC) {
const secs = ms / MS_IN_SEC;
str += `${secs.toFixed(2)}s`;
} else if (ms > 0 || !str.length) {
str += `${ms}ms`;
}
return str;
}
function itemChildren(item) { return item.c; }
function itemStart(item) { return item.s; }
function itemEnd(item) { return item.s + item.d; }
function itemDuration(item) { return item.d; }
function itemValues(item) { return item.v; }
function itemLevel(item) { return item.l; }
function itemLabel(item) { return item.v?.l; }
function itemType(item) { return item.v?.t; }
function itemError(item) { return item.e; }
function itemForcedFinish(item) { return item.f; }
function itemRef(item) { return item.v?.ref; }
function itemRefSource(item) { return item.v?.refId; }
function itemShortErrorMessage(item) {
if (itemError(item)) {
const e = itemError(item);
return e.name || e.stack.substr(0, e.stack.indexOf("\n"));
}
}
function itemCaption(item) {
if (itemType(item) === "network") {
return `${itemValues(item)?.method} ${itemValues(item)?.url}`;
} else if (itemLabel(item) && itemValues(item)?.id) {
return `${itemLabel(item)} ${itemValues(item).id}`;
} else if (itemLabel(item) && itemValues(item)?.status) {
return `${itemLabel(item)} (${itemValues(item).status})`;
} else if (itemLabel(item) && itemError(item)) {
return `${itemLabel(item)} (${itemShortErrorMessage(item)})`;
} else if (itemRef(item)) {
const refItem = itemByRef.get(itemRef(item));
if (refItem) {
return `ref "${itemCaption(refItem)}"`
} else {
return `unknown ref ${itemRef(item)}`
}
} else {
return itemLabel(item) || itemType(item);
}
}
function normalizeValueKey(key) {
switch (key) {
case "t": return "type";
case "l": return "label";
default: return key;
}
}
// returns the node and the total range (recursively) occupied by the node
function itemToNode(item) {
const hasChildren = !!itemChildren(item)?.length;
const className = {
item: true,
"has-children": hasChildren,
error: itemError(item),
[`type-${itemType(item)}`]: !!itemType(item),
[`level-${itemLevel(item)}`]: true,
};
const id = item.id;
let captionNode;
if (itemRef(item)) {
const refItem = itemByRef.get(itemRef(item));
if (refItem) {
captionNode = ["ref ", t.a({href: `#${refItem.id}`}, itemCaption(refItem))];
}
}
if (!captionNode) {
captionNode = itemCaption(item);
}
const li = t.li([
t.div([
hasChildren ? t.button({className: "toggleExpanded"}) : "",
t.a({className, id, href: `#${id}`}, [
t.span({class: "caption"}, captionNode),
t.span({class: "duration"}, `(${formatTime(itemDuration(item))})`),
])
])
]);
if (itemChildren(item) && itemChildren(item).length) {
li.appendChild(t.ol(itemChildren(item).map(item => {
return itemToNode(item);
})));
}
return li;
}
const highlightForm = document.getElementById("highlightForm");
highlightForm.addEventListener("submit", evt => {
evt.preventDefault();
const matchesOutput = document.getElementById("highlightMatches");
const query = document.getElementById("highlight").value;
if (query) {
matchesOutput.innerText = "Searching…";
let matches = 0;
processRecursively(rootItem, item => {
let domNode = document.getElementById(item.id);
if (itemMatchesFilter(item, query)) {
matches += 1;
domNode.classList.add("highlighted");
domNode = domNode.parentElement;
while (domNode.nodeName !== "SECTION") {
if (domNode.nodeName === "LI") {
domNode.classList.add("expanded");
}
domNode = domNode.parentElement;
}
} else {
domNode.classList.remove("highlighted");
}
});
matchesOutput.innerText = `${matches} matches`;
} else {
for (const node of document.querySelectorAll(".highlighted")) {
node.classList.remove("highlighted");
}
matchesOutput.innerText = "";
}
});
function itemMatchesFilter(item, query) {
if (itemError(item)) {
if (valueMatchesQuery(itemError(item), query)) {
return true;
}
}
return valueMatchesQuery(itemValues(item), query);
}
function valueMatchesQuery(value, query) {
if (typeof value === "string") {
return value.includes(query);
} else if (typeof value === "object" && value !== null) {
for (const key in value) {
if (value.hasOwnProperty(key) && valueMatchesQuery(value[key], query)) {
return true;
}
}
} else if (typeof value === "number") {
return value.toString().includes(query);
}
return false;
}
function processRecursively(item, callback, parentItem) {
if (item.id) {
callback(item, parentItem);
}
if (itemChildren(item)) {
for (let i = 0; i < itemChildren(item).length; i += 1) {
// do it in advance for a child as we don't want to do it for the rootItem
const child = itemChildren(item)[i];
processRecursively(child, callback, item);
}
}
}
document.getElementById("collapseAll").addEventListener("click", () => {
for (const node of document.querySelectorAll(".expanded")) {
node.classList.remove("expanded");
}
});
document.getElementById("hideCollapsed").addEventListener("click", () => {
for (const node of document.querySelectorAll("section > div.timeline > ol > li:not(.expanded)")) {
node.closest("section").classList.add("hidden");
}
});
document.getElementById("hideHighlightedSiblings").addEventListener("click", () => {
for (const node of document.querySelectorAll(".highlighted")) {
const list = node.closest("ol");
const siblings = Array.from(list.querySelectorAll("li > div > a:not(.highlighted)")).map(n => n.closest("li"));
for (const sibling of siblings) {
if (!sibling.classList.contains("expanded")) {
sibling.classList.add("hidden");
}
}
}
});
document.getElementById("showAll").addEventListener("click", () => {
for (const node of document.querySelectorAll(".hidden")) {
node.classList.remove("hidden");
}
});

8
scripts/postcss/css-url-to-variables.js
View File

@ -24,7 +24,13 @@ const idToPrepend = "icon-url";
function findAndReplaceUrl(decl, urlVariables, counter) {
const value = decl.value;
const parsed = valueParser(value);
let parsed;
try {
parsed = valueParser(value);
} catch (err) {
console.log(`Error trying to parse ${decl}`);
throw err;
}
parsed.walk(node => {
if (node.type !== "function" || node.value !== "url") {
return;

23
src/domain/AvatarSource.ts Normal file
View File

@ -0,0 +1,23 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Copyright 2020, 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
export interface AvatarSource {
get avatarLetter(): string;
get avatarColorNumber(): number;
avatarUrl(size: number): string | undefined;
get avatarTitle(): string;
}

72
src/domain/ErrorReportViewModel.ts Normal file
View File

@ -0,0 +1,72 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { ViewModel } from "./ViewModel";
import type { Options as BaseOptions } from "./ViewModel";
import type { Session } from "../matrix/Session";
import { ErrorViewModel } from "./ErrorViewModel";
import type { LogCallback, LabelOrValues } from "../logging/types";
export type Options<N extends object> = BaseOptions<N> & {
session: Session
};
/** Base class for view models that need to report errors to the UI. */
export class ErrorReportViewModel<N extends object, O extends Options<N> = Options<N>> extends ViewModel<N, O> {
private _errorViewModel?: ErrorViewModel<N>;
get errorViewModel(): ErrorViewModel<N> | undefined {
return this._errorViewModel;
}
/** Typically you'd want to use `logAndCatch` when implementing a view model method.
* Use `reportError` when showing errors on your model that were set by
* background processes using `ErrorBoundary` or you have some other
* special low-level need to write your try/catch yourself. */
protected reportError(error: Error) {
if (this._errorViewModel?.error === error) {
return;
}
this.disposeTracked(this._errorViewModel);
this._errorViewModel = this.track(new ErrorViewModel(this.childOptions({
error,
onClose: () => {
this._errorViewModel = this.disposeTracked(this._errorViewModel);
this.emitChange("errorViewModel");
}
})));
this.emitChange("errorViewModel");
}
/** Combines logging and error reporting in one method.
* Wrap the implementation of public view model methods
* with this to ensure errors are logged and reported.*/
protected logAndCatch<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, errorValue: T = undefined as unknown as T): T {
try {
let result = this.logger.run(labelOrValues, callback);
if (result instanceof Promise) {
result = result.catch(err => {
this.reportError(err);
return errorValue;
}) as unknown as T;
}
return result;
} catch (err) {
this.reportError(err);
return errorValue;
}
}
}

49
src/domain/ErrorViewModel.ts Normal file
View File

@ -0,0 +1,49 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { ViewModel, Options as BaseOptions } from "./ViewModel";
import {submitLogsFromSessionToDefaultServer} from "./rageshake";
import type { Session } from "../matrix/Session";
import type {SegmentType} from "./navigation/index";
type Options<N extends object> = {
error: Error
session: Session,
onClose: () => void
} & BaseOptions<N>;
export class ErrorViewModel<N extends object = SegmentType, O extends Options<N> = Options<N>> extends ViewModel<N, O> {
get message(): string {
return this.error.message;
}
get error(): Error {
return this.getOption("error");
}
close() {
this.getOption("onClose")();
}
async submitLogs(): Promise<boolean> {
try {
await submitLogsFromSessionToDefaultServer(this.getOption("session"), this.platform);
return true;
} catch (err) {
return false;
}
}
}

2
src/domain/RootViewModel.js
View File

@ -158,7 +158,7 @@ export class RootViewModel extends ViewModel {
}
_showSessionLoader(sessionId) {
const client = new Client(this.platform);
const client = new Client(this.platform, this.features);
client.startWithExistingSession(sessionId);
this._setSection(() => {
this._sessionLoadViewModel = new SessionLoadViewModel(this.childOptions({

10
src/domain/ViewModel.ts
View File

@ -30,6 +30,7 @@ import type {Navigation} from "./navigation/Navigation";
import type {SegmentType} from "./navigation/index";
import type {IURLRouter} from "./navigation/URLRouter";
import type { ITimeFormatter } from "../platform/types/types";
import type { FeatureSet } from "../features";
export type Options<T extends object = SegmentType> = {
platform: Platform;
@ -37,6 +38,7 @@ export type Options<T extends object = SegmentType> = {
urlRouter: IURLRouter<T>;
navigation: Navigation<T>;
emitChange?: (params: any) => void;
features: FeatureSet
}
@ -50,7 +52,7 @@ export class ViewModel<N extends object = SegmentType, O extends Options<N> = Op
this._options = options;
}
childOptions<T extends Object>(explicitOptions: T): T & Options<N> {
childOptions<T extends Object>(explicitOptions: T): T & O {
return Object.assign({}, this._options, explicitOptions);
}
@ -118,7 +120,7 @@ export class ViewModel<N extends object = SegmentType, O extends Options<N> = Op
return result;
}
emitChange(changedProps: any): void {
emitChange(changedProps?: any): void {
if (this._options.emitChange) {
this._options.emitChange(changedProps);
} else {
@ -142,6 +144,10 @@ export class ViewModel<N extends object = SegmentType, O extends Options<N> = Op
return this._options.urlRouter;
}
get features(): FeatureSet {
return this._options.features;
}
get navigation(): Navigation<N> {
// typescript needs a little help here
return this._options.navigation as unknown as Navigation<N>;

12
src/domain/avatar.ts
View File

@ -51,10 +51,18 @@ export function getIdentifierColorNumber(id: string): number {
return (hashCode(id) % 8) + 1;
}
export function getAvatarHttpUrl(avatarUrl: string, cssSize: number, platform: Platform, mediaRepository: MediaRepository): string | null {
export function getAvatarHttpUrl(avatarUrl: string | undefined, cssSize: number, platform: Platform, mediaRepository: MediaRepository): string | undefined {
if (avatarUrl) {
const imageSize = cssSize * platform.devicePixelRatio;
return mediaRepository.mxcUrlThumbnail(avatarUrl, imageSize, imageSize, "crop");
}
return null;
return undefined;
}
// move to AvatarView.js when converting to typescript
export interface IAvatarContract {
avatarLetter: string;
avatarColorNumber: number;
avatarUrl: (size: number) => string | undefined;
avatarTitle: string;
}

2
src/domain/login/LoginViewModel.ts
View File

@ -55,7 +55,7 @@ export class LoginViewModel extends ViewModel<SegmentType, Options> {
const {ready, defaultHomeserver, loginToken} = options;
this._ready = ready;
this._loginToken = loginToken;
this._client = new Client(this.platform);
this._client = new Client(this.platform, this.features);
this._homeserver = defaultHomeserver;
this._initViewModels();
}

3
src/domain/navigation/Navigation.ts
View File

@ -14,8 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue, ObservableValue} from "../../observable/ObservableValue";
import {BaseObservableValue, ObservableValue} from "../../observable/value";
type AllowsChild<T> = (parent: Segment<T> | undefined, child: Segment<T>) => boolean;

33
src/domain/rageshake.ts
View File

@ -16,11 +16,15 @@ limitations under the License.
import type {BlobHandle} from "../platform/web/dom/BlobHandle";
import type {RequestFunction} from "../platform/types/types";
import type {Platform} from "../platform/web/Platform";
import type {ILogger} from "../logging/types";
import type { IDBLogPersister } from "../logging/IDBLogPersister";
import type { Session } from "../matrix/Session";
// see https://github.com/matrix-org/rageshake#readme
type RageshakeData = {
// A textual description of the problem. Included in the details.log.gz file.
text: string | undefined;
text?: string;
// Application user-agent. Included in the details.log.gz file.
userAgent: string;
// Identifier for the application (eg 'riot-web'). Should correspond to a mapping configured in the configuration file for github issue reporting to work.
@ -28,7 +32,7 @@ type RageshakeData = {
// Application version. Included in the details.log.gz file.
version: string;
// Label to attach to the github issue, and include in the details file.
label: string | undefined;
label?: string;
};
export async function submitLogsToRageshakeServer(data: RageshakeData, logsBlob: BlobHandle, submitUrl: string, request: RequestFunction): Promise<void> {
@ -63,3 +67,28 @@ export async function submitLogsToRageshakeServer(data: RageshakeData, logsBlob:
// we don't bother with reading report_url from the body as the rageshake server doesn't always return it
// and would have to have CORS setup properly for us to be able to read it.
}
/** @throws {Error} */
export async function submitLogsFromSessionToDefaultServer(session: Session, platform: Platform): Promise<void> {
const {bugReportEndpointUrl} = platform.config;
if (!bugReportEndpointUrl) {
throw new Error("no server configured to submit logs");
}
const logReporters = (platform.logger as ILogger).reporters;
const exportReporter = logReporters.find(r => !!r["export"]) as IDBLogPersister | undefined;
if (!exportReporter) {
throw new Error("No logger that can export configured");
}
const logExport = await exportReporter.export();
await submitLogsToRageshakeServer(
{
app: "hydrogen",
userAgent: platform.description,
version: platform.version,
text: `Submit logs from settings for user ${session.userId} on device ${session.deviceId}`,
},
logExport.asBlob(),
bugReportEndpointUrl,
platform.request
);
}

2
src/domain/session/RoomGridViewModel.js
View File

@ -186,7 +186,7 @@ export class RoomGridViewModel extends ViewModel {
}
import {createNavigation} from "../navigation/index";
import {ObservableValue} from "../../observable/ObservableValue";
import {ObservableValue} from "../../observable/value";
export function tests() {
class RoomVMMock {

2
src/domain/session/RoomViewModelObservable.js
View File

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableValue} from "../../observable/ObservableValue";
import {ObservableValue} from "../../observable/value";
import {RoomStatus} from "../../matrix/room/common";
/**

17
src/domain/session/SessionViewModel.js
View File

@ -30,6 +30,7 @@ import {ViewModel} from "../ViewModel";
import {RoomViewModelObservable} from "./RoomViewModelObservable.js";
import {RightPanelViewModel} from "./rightpanel/RightPanelViewModel.js";
import {SyncStatus} from "../../matrix/Sync.js";
import {ToastCollectionViewModel} from "./toast/ToastCollectionViewModel";
export class SessionViewModel extends ViewModel {
constructor(options) {
@ -47,6 +48,9 @@ export class SessionViewModel extends ViewModel {
this._gridViewModel = null;
this._createRoomViewModel = null;
this._joinRoomViewModel = null;
this._toastCollectionViewModel = this.track(new ToastCollectionViewModel(this.childOptions({
session: this._client.session,
})));
this._setupNavigation();
this._setupForcedLogoutOnAccessTokenInvalidation();
}
@ -126,6 +130,11 @@ export class SessionViewModel extends ViewModel {
start() {
this._sessionStatusViewModel.start();
if (this.features.calls) {
this._client.session.callHandler.loadCalls("m.ring");
// TODO: only do this when opening the room
this._client.session.callHandler.loadCalls("m.prompt");
}
}
get activeMiddleViewModel() {
@ -170,6 +179,10 @@ export class SessionViewModel extends ViewModel {
return this._joinRoomViewModel;
}
get toastCollectionViewModel() {
return this._toastCollectionViewModel;
}
_updateGrid(roomIds) {
const changed = !(this._gridViewModel && roomIds);
const currentRoomId = this.navigation.path.get("room");
@ -211,7 +224,7 @@ export class SessionViewModel extends ViewModel {
_createRoomViewModelInstance(roomId) {
const room = this._client.session.rooms.get(roomId);
if (room) {
const roomVM = new RoomViewModel(this.childOptions({room}));
const roomVM = new RoomViewModel(this.childOptions({room, session: this._client.session}));
roomVM.load();
return roomVM;
}
@ -228,7 +241,7 @@ export class SessionViewModel extends ViewModel {
async _createArchivedRoomViewModel(roomId) {
const room = await this._client.session.loadArchivedRoom(roomId);
if (room) {
const roomVM = new RoomViewModel(this.childOptions({room}));
const roomVM = new RoomViewModel(this.childOptions({room, session: this._client.session}));
roomVM.load();
return roomVM;
}

288
src/domain/session/room/CallViewModel.ts Normal file
View File

@ -0,0 +1,288 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AvatarSource} from "../../AvatarSource";
import type {ViewModel} from "../../ViewModel";
import {ErrorReportViewModel, Options as BaseOptions} from "../../ErrorReportViewModel";
import {getStreamVideoTrack, getStreamAudioTrack} from "../../../matrix/calls/common";
import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar";
import {EventObservableValue} from "../../../observable/value";
import {ObservableValueMap, BaseObservableMap} from "../../../observable/map";
import {ErrorViewModel} from "../../ErrorViewModel";
import type {Room} from "../../../matrix/room/Room";
import type {GroupCall} from "../../../matrix/calls/group/GroupCall";
import type {Member} from "../../../matrix/calls/group/Member";
import type {RoomMember} from "../../../matrix/room/members/RoomMember";
import type {BaseObservableList} from "../../../observable/list/BaseObservableList";
import type {BaseObservableValue} from "../../../observable/value";
import type {Stream} from "../../../platform/types/MediaDevices";
import type {MediaRepository} from "../../../matrix/net/MediaRepository";
import type {Session} from "../../../matrix/Session";
import type {SegmentType} from "../../navigation";
type Options<N extends object> = BaseOptions<N> & {
call: GroupCall,
room: Room,
};
export class CallViewModel extends ErrorReportViewModel<SegmentType, Options<SegmentType>> {
public readonly memberViewModels: BaseObservableList<IStreamViewModel>;
constructor(options: Options<SegmentType>) {
super(options);
const callObservable = new EventObservableValue(this.call, "change");
this.track(callObservable.subscribe(() => this.onUpdate()));
const ownMemberViewModelMap = new ObservableValueMap("self", callObservable)
.mapValues((call, emitChange) => new OwnMemberViewModel(this.childOptions({call, emitChange})), () => {});
const otherMemberViewModels = this.call.members
.filterValues(member => member.isConnected)
.mapValues(
(member, emitChange) => new CallMemberViewModel(this.childOptions({
member,
emitChange,
mediaRepository: this.getOption("room").mediaRepository
})),
(param, vm) => vm?.onUpdate(),
) as BaseObservableMap<string, IStreamViewModel>;
this.memberViewModels = otherMemberViewModels
.join(ownMemberViewModelMap)
.sortValues((a, b) => a.compare(b));
this.track(this.memberViewModels.subscribe({
onRemove: () => {
this.emitChange(); // update memberCount
},
onAdd: () => {
this.emitChange(); // update memberCount
},
onUpdate: () => {},
onReset: () => {},
onMove: () => {}
}))
}
get isCameraMuted(): boolean {
return this.call.muteSettings?.camera ?? true;
}
get isMicrophoneMuted(): boolean {
return this.call.muteSettings?.microphone ?? true;
}
get memberCount(): number {
return this.memberViewModels.length;
}
get name(): string {
return this.call.name;
}
get id(): string {
return this.call.id;
}
private get call(): GroupCall {
return this.getOption("call");
}
private onUpdate() {
if (this.call.error) {
this.reportError(this.call.error);
}
}
async hangup() {
this.logAndCatch("CallViewModel.hangup", async log => {
if (this.call.hasJoined) {
await this.call.leave(log);
}
});
}
async toggleCamera() {
this.logAndCatch("Call.toggleCamera", async log => {
const {localMedia, muteSettings} = this.call;
if (muteSettings && localMedia) {
// unmute but no track?
if (muteSettings.camera && !getStreamVideoTrack(localMedia.userMedia)) {
const stream = await this.platform.mediaDevices.getMediaTracks(!muteSettings.microphone, true);
await this.call.setMedia(localMedia.withUserMedia(stream));
} else {
await this.call.setMuted(muteSettings.toggleCamera());
}
this.emitChange();
}
});
}
async toggleMicrophone() {
this.logAndCatch("Call.toggleMicrophone", async log => {
const {localMedia, muteSettings} = this.call;
if (muteSettings && localMedia) {
// unmute but no track?
if (muteSettings.microphone && !getStreamAudioTrack(localMedia.userMedia)) {
const stream = await this.platform.mediaDevices.getMediaTracks(true, !muteSettings.camera);
await this.call.setMedia(localMedia.withUserMedia(stream));
} else {
await this.call.setMuted(muteSettings.toggleMicrophone());
}
this.emitChange();
}
});
}
}
class OwnMemberViewModel extends ErrorReportViewModel<SegmentType, Options<SegmentType>> implements IStreamViewModel {
private memberObservable: undefined | BaseObservableValue<RoomMember>;
constructor(options: Options<SegmentType>) {
super(options);
this.init();
}
async init() {
const room = this.getOption("room");
this.memberObservable = await room.observeMember(room.user.id);
this.track(this.memberObservable!.subscribe(() => {
this.emitChange(undefined);
}));
}
get errorViewModel(): ErrorViewModel | undefined {
return undefined;
}
get stream(): Stream | undefined {
return this.call.localPreviewMedia?.userMedia;
}
private get call(): GroupCall {
return this.getOption("call");
}
get isCameraMuted(): boolean {
return this.call.muteSettings?.camera ?? true;
}
get isMicrophoneMuted(): boolean {
return this.call.muteSettings?.microphone ?? true;
}
get avatarLetter(): string {
const member = this.memberObservable?.get();
if (member) {
return avatarInitials(member.name);
} else {
return this.getOption("room").user.id;
}
}
get avatarColorNumber(): number {
return getIdentifierColorNumber(this.getOption("room").user.id);
}
avatarUrl(size: number): string | undefined {
const member = this.memberObservable?.get();
if (member) {
return getAvatarHttpUrl(member.avatarUrl, size, this.platform, this.getOption("room").mediaRepository);
}
}
get avatarTitle(): string {
const member = this.memberObservable?.get();
if (member) {
return member.name;
} else {
return this.getOption("room").user.id;
}
}
compare(other: IStreamViewModel): number {
// I always come first.
return -1;
}
}
type MemberOptions<N extends object> = BaseOptions<N> & {
member: Member,
mediaRepository: MediaRepository,
};
export class CallMemberViewModel extends ErrorReportViewModel<SegmentType, MemberOptions<SegmentType>> implements IStreamViewModel {
get stream(): Stream | undefined {
return this.member.remoteMedia?.userMedia;
}
private get member(): Member {
return this.getOption("member");
}
get isCameraMuted(): boolean {
return this.member.remoteMuteSettings?.camera ?? true;
}
get isMicrophoneMuted(): boolean {
return this.member.remoteMuteSettings?.microphone ?? true;
}
get avatarLetter(): string {
return avatarInitials(this.member.member.name);
}
get avatarColorNumber(): number {
return getIdentifierColorNumber(this.member.userId);
}
avatarUrl(size: number): string | undefined {
const {avatarUrl} = this.member.member;
const mediaRepository = this.getOption("mediaRepository");
return getAvatarHttpUrl(avatarUrl, size, this.platform, mediaRepository);
}
get avatarTitle(): string {
return this.member.member.name;
}
onUpdate() {
this.mapMemberSyncErrorIfNeeded();
}
private mapMemberSyncErrorIfNeeded() {
if (this.member.error) {
this.reportError(this.member.error);
}
}
compare(other: IStreamViewModel): number {
if (other instanceof CallMemberViewModel) {
const myUserId = this.member.member.userId;
const otherUserId = other.member.member.userId;
if(myUserId === otherUserId) {
return 0;
}
return myUserId < otherUserId ? -1 : 1;
} else {
return -other.compare(this);
}
}
}
export interface IStreamViewModel extends AvatarSource, ViewModel {
get stream(): Stream | undefined;
get isCameraMuted(): boolean;
get isMicrophoneMuted(): boolean;
get errorViewModel(): ErrorViewModel | undefined;
compare(other: IStreamViewModel): number;
}

236
src/domain/session/room/RoomViewModel.js
View File

@ -17,15 +17,19 @@ limitations under the License.
import {TimelineViewModel} from "./timeline/TimelineViewModel.js";
import {ComposerViewModel} from "./ComposerViewModel.js"
import {CallViewModel} from "./CallViewModel"
import {PickMapObservableValue} from "../../../observable/value";
import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar";
import {ErrorReportViewModel} from "../../ErrorReportViewModel";
import {ViewModel} from "../../ViewModel";
import {imageToInfo} from "../common.js";
import {LocalMedia} from "../../../matrix/calls/LocalMedia";
// TODO: remove fallback so default isn't included in bundle for SDK users that have their custom tileClassForEntry
// this is a breaking SDK change though to make this option mandatory
import {tileClassForEntry as defaultTileClassForEntry} from "./timeline/tiles/index";
import {joinRoom} from "../../../matrix/room/joinRoom";
export class RoomViewModel extends ViewModel {
export class RoomViewModel extends ErrorReportViewModel {
constructor(options) {
super(options);
const {room, tileClassForEntry} = options;
@ -34,8 +38,6 @@ export class RoomViewModel extends ViewModel {
this._tileClassForEntry = tileClassForEntry ?? defaultTileClassForEntry;
this._tileOptions = undefined;
this._onRoomChange = this._onRoomChange.bind(this);
this._timelineError = null;
this._sendError = null;
this._composerVM = null;
if (room.isArchived) {
this._composerVM = this.track(new ArchivedViewModel(this.childOptions({archivedRoom: room})));
@ -44,13 +46,42 @@ export class RoomViewModel extends ViewModel {
}
this._clearUnreadTimout = null;
this._closeUrl = this.urlRouter.urlUntilSegment("session");
this._setupCallViewModel();
}
_setupCallViewModel() {
if (!this.features.calls) {
return;
}
// pick call for this room with lowest key
const calls = this.getOption("session").callHandler.calls;
this._callObservable = new PickMapObservableValue(calls.filterValues(c => {
return c.roomId === this._room.id && c.hasJoined;
}));
this._callViewModel = undefined;
this.track(this._callObservable.subscribe(call => {
if (call && this._callViewModel && call.id === this._callViewModel.id) {
return;
}
this._callViewModel = this.disposeTracked(this._callViewModel);
if (call) {
this._callViewModel = this.track(new CallViewModel(this.childOptions({call, room: this._room})));
}
this.emitChange("callViewModel");
}));
const call = this._callObservable.get();
// TODO: cleanup this duplication to create CallViewModel
if (call) {
this._callViewModel = this.track(new CallViewModel(this.childOptions({call, room: this._room})));
}
}
async load() {
this._room.on("change", this._onRoomChange);
try {
const timeline = await this._room.openTimeline();
this.logAndCatch("RoomViewModel.load", async log => {
this._room.on("change", this._onRoomChange);
const timeline = await this._room.openTimeline(log);
this._tileOptions = this.childOptions({
session: this.getOption("session"),
roomVM: this,
timeline,
tileClassForEntry: this._tileClassForEntry,
@ -60,12 +91,8 @@ export class RoomViewModel extends ViewModel {
timeline,
})));
this.emitChange("timelineViewModel");
} catch (err) {
console.error(`room.openTimeline(): ${err.message}:\n${err.stack}`);
this._timelineError = err;
this.emitChange("error");
}
this._clearUnreadAfterDelay();
await this._clearUnreadAfterDelay(log);
});
}
async _recreateComposerOnPowerLevelChange() {
@ -92,24 +119,28 @@ export class RoomViewModel extends ViewModel {
recreateComposer(oldCanSendMessage);
}
async _clearUnreadAfterDelay() {
async _clearUnreadAfterDelay(log) {
if (this._room.isArchived || this._clearUnreadTimout) {
return;
}
this._clearUnreadTimout = this.clock.createTimeout(2000);
try {
await this._clearUnreadTimout.elapsed();
await this._room.clearUnread();
await this._room.clearUnread(log);
this._clearUnreadTimout = null;
} catch (err) {
if (err.name !== "AbortError") {
if (err.name === "AbortError") {
log.set("clearUnreadCancelled", true);
} else {
throw err;
}
}
}
focus() {
this._clearUnreadAfterDelay();
this.logAndCatch("RoomViewModel.focus", async log => {
this._clearUnreadAfterDelay(log);
});
}
dispose() {
@ -139,16 +170,6 @@ export class RoomViewModel extends ViewModel {
get timelineViewModel() { return this._timelineVM; }
get isEncrypted() { return this._room.isEncrypted; }
get error() {
if (this._timelineError) {
return `Something went wrong loading the timeline: ${this._timelineError.message}`;
}
if (this._sendError) {
return `Something went wrong sending your message: ${this._sendError.message}`;
}
return "";
}
get avatarLetter() {
return avatarInitials(this.name);
}
@ -191,26 +212,51 @@ export class RoomViewModel extends ViewModel {
_createTile(entry) {
if (this._tileOptions) {
const Tile = this._tileOptions.tileClassForEntry(entry);
const Tile = this._tileOptions.tileClassForEntry(entry, this._tileOptions);
if (Tile) {
return new Tile(entry, this._tileOptions);
}
}
}
_sendMessage(message, replyingTo) {
return this.logAndCatch("RoomViewModel.sendMessage", async log => {
let success = false;
if (!this._room.isArchived && message) {
let msgtype = "m.text";
if (message.startsWith("//")) {
message = message.substring(1).trim();
} else if (message.startsWith("/")) {
const result = await this._processCommand(message);
msgtype = result.msgtype;
message = result.message;
}
let content;
if (replyingTo) {
log.set("replyingTo", replyingTo.eventId);
content = await replyingTo.createReplyContent(msgtype, message);
} else {
content = {msgtype, body: message};
}
await this._room.sendEvent("m.room.message", content, undefined, log);
success = true;
}
log.set("success", success);
return success;
}, false);
}
async _processCommandJoin(roomName) {
try {
const session = this._options.client.session;
const roomId = await joinRoom(roomName, session);
this.navigation.push("room", roomId);
} catch (err) {
this._sendError = err;
this._timelineError = null;
this.emitChange("error");
this.reportError(err);
}
}
async _processCommand (message) {
async _processCommand(message) {
let msgtype;
const [commandName, ...args] = message.substring(1).split(" ");
switch (commandName) {
@ -223,9 +269,7 @@ export class RoomViewModel extends ViewModel {
const roomName = args[0];
await this._processCommandJoin(roomName);
} else {
this._sendError = new Error("join syntax: /join <room-id>");
this._timelineError = null;
this.emitChange("error");
this.reportError(new Error("join syntax: /join <room-id>"));
}
break;
case "shrug":
@ -245,78 +289,44 @@ export class RoomViewModel extends ViewModel {
msgtype = "m.text";
break;
default:
this._sendError = new Error(`no command name "${commandName}". To send the message instead of executing, please type "/${message}"`);
this._timelineError = null;
this.emitChange("error");
this.reportError(new Error(`no command name "${commandName}". To send the message instead of executing, please type "/${message}"`));
message = undefined;
}
return {type: msgtype, message: message};
}
async _sendMessage(message, replyingTo) {
if (!this._room.isArchived && message) {
let messinfo = {type : "m.text", message : message};
if (message.startsWith("//")) {
messinfo.message = message.substring(1).trim();
} else if (message.startsWith("/")) {
messinfo = await this._processCommand(message);
}
try {
const msgtype = messinfo.type;
const message = messinfo.message;
if (msgtype && message) {
if (replyingTo) {
await replyingTo.reply(msgtype, message);
} else {
await this._room.sendEvent("m.room.message", {msgtype, body: message});
}
}
} catch (err) {
console.error(`room.sendMessage(): ${err.message}:\n${err.stack}`);
this._sendError = err;
this._timelineError = null;
this.emitChange("error");
return false;
}
return true;
}
return false;
return {type: msgtype, message: message};
}
async _pickAndSendFile() {
try {
_pickAndSendFile() {
return this.logAndCatch("RoomViewModel.sendFile", async log => {
const file = await this.platform.openFile();
if (!file) {
log.set("cancelled", true);
return;
}
return this._sendFile(file);
} catch (err) {
console.error(err);
}
return this._sendFile(file, log);
});
}
async _sendFile(file) {
async _sendFile(file, log) {
const content = {
body: file.name,
msgtype: "m.file"
};
await this._room.sendEvent("m.room.message", content, {
"url": this._room.createAttachment(file.blob, file.name)
});
}, log);
}
async _pickAndSendVideo() {
try {
_pickAndSendVideo() {
return this.logAndCatch("RoomViewModel.sendVideo", async log => {
if (!this.platform.hasReadPixelPermission()) {
alert("Please allow canvas image data access, so we can scale your images down.");
return;
throw new Error("Please allow canvas image data access, so we can scale your images down.");
}
const file = await this.platform.openFile("video/*");
if (!file) {
return;
}
if (!file.blob.mimeType.startsWith("video/")) {
return this._sendFile(file);
return this._sendFile(file, log);
}
let video;
try {
@ -344,26 +354,23 @@ export class RoomViewModel extends ViewModel {
content.info.thumbnail_info = imageToInfo(thumbnail);
attachments["info.thumbnail_url"] =
this._room.createAttachment(thumbnail.blob, file.name);
await this._room.sendEvent("m.room.message", content, attachments);
} catch (err) {
this._sendError = err;
this.emitChange("error");
console.error(err.stack);
}
await this._room.sendEvent("m.room.message", content, attachments, log);
});
}
async _pickAndSendPicture() {
try {
this.logAndCatch("RoomViewModel.sendPicture", async log => {
if (!this.platform.hasReadPixelPermission()) {
alert("Please allow canvas image data access, so we can scale your images down.");
return;
}
const file = await this.platform.openFile("image/*");
if (!file) {
log.set("cancelled", true);
return;
}
if (!file.blob.mimeType.startsWith("image/")) {
return this._sendFile(file);
return this._sendFile(file, log);
}
let image = await this.platform.loadImage(file.blob);
const limit = await this.platform.settingsStorage.getInt("sentImageSizeLimit");
@ -386,12 +393,8 @@ export class RoomViewModel extends ViewModel {
attachments["info.thumbnail_url"] =
this._room.createAttachment(thumbnail.blob, file.name);
}
await this._room.sendEvent("m.room.message", content, attachments);
} catch (err) {
this._sendError = err;
this.emitChange("error");
console.error(err.stack);
}
await this._room.sendEvent("m.room.message", content, attachments, log);
});
}
get room() {
@ -402,6 +405,10 @@ export class RoomViewModel extends ViewModel {
return this._composerVM;
}
get callViewModel() {
return this._callViewModel;
}
openDetailsPanel() {
let path = this.navigation.path.until("room");
path = path.with(this.navigation.segment("right-panel", true));
@ -414,10 +421,41 @@ export class RoomViewModel extends ViewModel {
this._composerVM.setReplyingTo(entry);
}
}
dismissError() {
this._sendError = null;
this.emitChange("error");
startCall() {
return this.logAndCatch("RoomViewModel.startCall", async log => {
if (!this.features.calls) {
log.set("feature_disbled", true);
return;
}
log.set("roomId", this._room.id);
let localMedia;
try {
const stream = await this.platform.mediaDevices.getMediaTracks(false, true);
localMedia = new LocalMedia().withUserMedia(stream);
} catch (err) {
throw new Error(`Could not get local audio and/or video stream: ${err.message}`);
}
const session = this.getOption("session");
let call;
try {
// this will set the callViewModel above as a call will be added to callHandler.calls
call = await session.callHandler.createCall(
this._room.id,
"m.video",
"A call " + Math.round(this.platform.random() * 100),
undefined,
log
);
} catch (err) {
throw new Error(`Could not create call: ${err.message}`);
}
try {
await call.join(localMedia, log);
} catch (err) {
throw new Error(`Could not join call: ${err.message}`);
}
});
}
}

2
src/domain/session/room/timeline/ReactionsViewModel.js
View File

@ -189,7 +189,7 @@ import {HomeServer as MockHomeServer} from "../../../../mocks/HomeServer.js";
// other imports
import {BaseMessageTile} from "./tiles/BaseMessageTile.js";
import {MappedList} from "../../../../observable/list/MappedList";
import {ObservableValue} from "../../../../observable/ObservableValue";
import {ObservableValue} from "../../../../observable/value";
import {PowerLevels} from "../../../../matrix/room/PowerLevels.js";
export function tests() {

2
src/domain/session/room/timeline/TilesCollection.js
View File

@ -34,7 +34,7 @@ export class TilesCollection extends BaseObservableList {
}
_createTile(entry) {
const Tile = this._tileOptions.tileClassForEntry(entry);
const Tile = this._tileOptions.tileClassForEntry(entry, this._tileOptions);
if (Tile) {
return new Tile(entry, this._tileOptions);
}

14
src/domain/session/room/timeline/tiles/BaseMessageTile.js
View File

@ -48,14 +48,6 @@ export class BaseMessageTile extends SimpleTile {
return `https://matrix.to/#/${encodeURIComponent(this.sender)}`;
}
get displayName() {
return this._entry.displayName || this.sender;
}
get sender() {
return this._entry.sender;
}
get memberPanelLink() {
return `${this.urlRouter.urlUntilSegment("room")}/member/${this.sender}`;
}
@ -134,7 +126,7 @@ export class BaseMessageTile extends SimpleTile {
if (action?.shouldReplace || !this._replyTile) {
this.disposeTracked(this._replyTile);
const tileClassForEntry = this._options.tileClassForEntry;
const ReplyTile = tileClassForEntry(replyEntry);
const ReplyTile = tileClassForEntry(replyEntry, this._options);
if (ReplyTile) {
this._replyTile = new ReplyTile(replyEntry, this._options);
}
@ -149,8 +141,8 @@ export class BaseMessageTile extends SimpleTile {
this._roomVM.startReply(this._entry);
}
reply(msgtype, body, log = null) {
return this._room.sendEvent("m.room.message", this._entry.reply(msgtype, body), null, log);
createReplyContent(msgtype, body) {
return this._entry.createReplyContent(msgtype, body);
}
redact(reason, log) {

180
src/domain/session/room/timeline/tiles/CallTile.js Normal file
View File

@ -0,0 +1,180 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {SimpleTile} from "./SimpleTile.js";
import {ViewModel} from "../../../../ViewModel";
import {LocalMedia} from "../../../../../matrix/calls/LocalMedia";
import {CallType} from "../../../../../matrix/calls/callEventTypes";
import {avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../../../avatar";
// TODO: timeline entries for state events with the same state key and type
// should also update previous entries in the timeline, so we can update the name of the call, whether it is terminated, etc ...
// alternatively, we could just subscribe to the GroupCall and spontanously emit an update when it updates
export class CallTile extends SimpleTile {
constructor(entry, options) {
super(entry, options);
const calls = this.getOption("session").callHandler.calls;
this._callSubscription = undefined;
this._memberSizeSubscription = undefined;
const call = calls.get(this._entry.stateKey);
if (call && !call.isTerminated) {
this._call = call;
this.memberViewModels = this._setupMembersList(this._call);
this._callSubscription = this.track(this._call.disposableOn("change", () => {
this._onCallUpdate();
}));
this._memberSizeSubscription = this.track(this._call.members.observeSize().subscribe(() => {
this.emitChange("memberCount");
}));
this._onCallUpdate();
}
}
_onCallUpdate() {
// unsubscribe when terminated
if (this._call.isTerminated) {
this._durationInterval = this.disposeTracked(this._durationInterval);
this._callSubscription = this.disposeTracked(this._callSubscription);
this._call = undefined;
} else if (!this._durationInterval) {
this._durationInterval = this.track(this.platform.clock.createInterval(() => {
this.emitChange("duration");
}, 1000));
}
this.emitChange();
}
_setupMembersList(call) {
return call.members.mapValues(
(member, emitChange) => new MemberAvatarViewModel(this.childOptions({
member,
emitChange,
mediaRepository: this.getOption("room").mediaRepository
})),
).sortValues((a, b) => a.userId.localeCompare(b.userId));
}
get memberCount() {
// TODO: emit updates for this property
if (this._call) {
return this._call.members.size;
}
return 0;
}
get confId() {
return this._entry.stateKey;
}
get duration() {
if (this._call && this._call.duration) {
return this.timeFormatter.formatDuration(this._call.duration);
} else {
return "";
}
}
get shape() {
return "call";
}
get canJoin() {
return this._call && !this._call.hasJoined && !this._call.usesFoci;
}
get canLeave() {
return this._call && this._call.hasJoined;
}
get title() {
if (this._call) {
if (this.type === CallType.Video) {
return `${this.displayName} started a video call`;
} else {
return `${this.displayName} started a voice call`;
}
} else {
if (this.type === CallType.Video) {
return `Video call ended`;
} else {
return `Voice call ended`;
}
}
}
get typeLabel() {
if (this._call && this._call.usesFoci) {
return `This call uses a stream-forwarding unit, which isn't supported yet, so you can't join this call.`;
}
if (this.type === CallType.Video) {
return `Video call`;
} else {
return `Voice call`;
}
}
get type() {
return this._entry.event.content["m.type"];
}
async join() {
await this.logAndCatch("CallTile.join", async log => {
if (this.canJoin) {
const stream = await this.platform.mediaDevices.getMediaTracks(false, true);
const localMedia = new LocalMedia().withUserMedia(stream);
await this._call.join(localMedia, log);
}
});
}
async leave() {
await this.logAndCatch("CallTile.leave", async log => {
if (this.canLeave) {
await this._call.leave(log);
}
});
}
}
class MemberAvatarViewModel extends ViewModel {
get _member() {
return this.getOption("member");
}
get userId() {
return this._member.userId;
}
get avatarLetter() {
return avatarInitials(this._member.member.name);
}
get avatarColorNumber() {
return getIdentifierColorNumber(this._member.userId);
}
avatarUrl(size) {
const {avatarUrl} = this._member.member;
const mediaRepository = this.getOption("mediaRepository");
return getAvatarHttpUrl(avatarUrl, size, this.platform, mediaRepository);
}
get avatarTitle() {
return this._member.member.name;
}
}

3
src/domain/session/room/timeline/tiles/ITile.ts
View File

@ -16,7 +16,8 @@ export enum TileShape {
MissingAttachment = "missing-attachment",
Redacted = "redacted",
Video = "video",
DateHeader = "date-header"
DateHeader = "date-header",
Call = "call",
}
// TODO: should we imply inheriting from view model here?

12
src/domain/session/room/timeline/tiles/SimpleTile.js
View File

@ -15,12 +15,12 @@ limitations under the License.
*/
import {UpdateAction} from "../UpdateAction.js";
import {ErrorReportViewModel} from "../../../../ErrorReportViewModel";
import {TileShape} from "./ITile";
import {ViewModel} from "../../../../ViewModel";
import {SendStatus} from "../../../../../matrix/room/sending/PendingEvent.js";
import {DateTile} from "./DateTile";
export class SimpleTile extends ViewModel {
export class SimpleTile extends ErrorReportViewModel {
constructor(entry, options) {
super(options);
this._entry = entry;
@ -187,6 +187,14 @@ export class SimpleTile extends ViewModel {
get _ownMember() {
return this._options.timeline.me;
}
get displayName() {
return this._entry.displayName || this.sender;
}
get sender() {
return this._entry.sender;
}
}
import { EventEntry } from "../../../../../matrix/room/timeline/entries/EventEntry.js";

13
src/domain/session/room/timeline/tiles/index.ts
View File

@ -26,9 +26,11 @@ import {RoomMemberTile} from "./RoomMemberTile.js";
import {EncryptedEventTile} from "./EncryptedEventTile.js";
import {EncryptionEnabledTile} from "./EncryptionEnabledTile.js";
import {MissingAttachmentTile} from "./MissingAttachmentTile.js";
import {CallTile} from "./CallTile.js";
import type {ITile, TileShape} from "./ITile";
import type {Room} from "../../../../../matrix/room/Room";
import type {Session} from "../../../../../matrix/Session";
import type {Timeline} from "../../../../../matrix/room/timeline/Timeline";
import type {FragmentBoundaryEntry} from "../../../../../matrix/room/timeline/entries/FragmentBoundaryEntry";
import type {EventEntry} from "../../../../../matrix/room/timeline/entries/EventEntry";
@ -38,13 +40,14 @@ import type {Options as ViewModelOptions} from "../../../../ViewModel";
export type TimelineEntry = FragmentBoundaryEntry | EventEntry | PendingEventEntry;
export type TileClassForEntryFn = (entry: TimelineEntry) => TileConstructor | undefined;
export type Options = ViewModelOptions & {
session: Session,
room: Room,
timeline: Timeline
tileClassForEntry: TileClassForEntryFn;
};
export type TileConstructor = new (entry: TimelineEntry, options: Options) => ITile;
export function tileClassForEntry(entry: TimelineEntry): TileConstructor | undefined {
export function tileClassForEntry(entry: TimelineEntry, options: Options): TileConstructor | undefined {
if (entry.isGap) {
return GapTile;
} else if (entry.isPending && entry.pendingEvent.isMissingAttachments) {
@ -86,6 +89,14 @@ export function tileClassForEntry(entry: TimelineEntry): TileConstructor | undef
return EncryptedEventTile;
case "m.room.encryption":
return EncryptionEnabledTile;
case "org.matrix.msc3401.call": {
// if prevContent is present, it's an update to a call event, which we don't render
// as the original event is updated through the call object which receive state event updates
if (options.features.calls && entry.stateKey && !entry.prevContent) {
return CallTile;
}
return undefined;
}
default:
// unknown type not rendered
return undefined;

70
src/domain/session/settings/FeaturesViewModel.ts Normal file
View File

@ -0,0 +1,70 @@
/*
Copyright 2020 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ViewModel} from "../../ViewModel";
import type {Options as BaseOptions} from "../../ViewModel";
import {FeatureFlag, FeatureSet} from "../../../features";
import type {SegmentType} from "../../navigation/index";
export class FeaturesViewModel extends ViewModel {
public readonly featureViewModels: ReadonlyArray<FeatureViewModel>;
constructor(options) {
super(options);
this.featureViewModels = [
new FeatureViewModel(this.childOptions({
name: this.i18n`Audio/video calls`,
description: this.i18n`Allows starting and participating in A/V calls compatible with Element Call (MSC3401). Look for the start call option in the room menu ((...) in the right corner) to start a call.`,
feature: FeatureFlag.Calls
})),
];
}
}
type FeatureOptions = BaseOptions & {
feature: FeatureFlag,
description: string,
name: string
};
export class FeatureViewModel extends ViewModel<SegmentType, FeatureOptions> {
get enabled(): boolean {
return this.features.isFeatureEnabled(this.getOption("feature"));
}
async enableFeature(enabled: boolean): Promise<void> {
let newFeatures;
if (enabled) {
newFeatures = this.features.withFeature(this.getOption("feature"));
} else {
newFeatures = this.features.withoutFeature(this.getOption("feature"));
}
await newFeatures.store(this.platform.settingsStorage);
this.platform.restart();
}
get id(): string {
return `${this.getOption("feature")}`;
}
get name(): string {
return this.getOption("name");
}
get description(): string {
return this.getOption("description");
}
}

5
src/domain/session/settings/KeyBackupViewModel.js
View File

@ -17,6 +17,7 @@ limitations under the License.
import {ViewModel} from "../../ViewModel";
import {KeyType} from "../../../matrix/ssss/index";
import {createEnum} from "../../../utils/enum";
import {FlatMapObservableValue} from "../../../observable/value";
export const Status = createEnum("Enabled", "SetupKey", "SetupPhrase", "Pending", "NewVersionAvailable");
export const BackupWriteStatus = createEnum("Writing", "Stopped", "Done", "Pending");
@ -29,8 +30,8 @@ export class KeyBackupViewModel extends ViewModel {
this._isBusy = false;
this._dehydratedDeviceId = undefined;
this._status = undefined;
this._backupOperation = this._session.keyBackup.flatMap(keyBackup => keyBackup.operationInProgress);
this._progress = this._backupOperation.flatMap(op => op.progress);
this._backupOperation = new FlatMapObservableValue(this._session.keyBackup, keyBackup => keyBackup.operationInProgress);
this._progress = new FlatMapObservableValue(this._backupOperation, op => op.progress);
this.track(this._backupOperation.subscribe(() => {
// see if needsNewKey might be set
this._reevaluateStatus();

46
src/domain/session/settings/SettingsViewModel.js
View File

@ -16,7 +16,8 @@ limitations under the License.
import {ViewModel} from "../../ViewModel";
import {KeyBackupViewModel} from "./KeyBackupViewModel.js";
import {submitLogsToRageshakeServer} from "../../../domain/rageshake";
import {FeaturesViewModel} from "./FeaturesViewModel";
import {submitLogsFromSessionToDefaultServer} from "../../../domain/rageshake";
class PushNotificationStatus {
constructor() {
@ -53,6 +54,7 @@ export class SettingsViewModel extends ViewModel {
this.pushNotifications = new PushNotificationStatus();
this._activeTheme = undefined;
this._logsFeedbackMessage = undefined;
this._featuresViewModel = new FeaturesViewModel(this.childOptions());
}
get _session() {
@ -125,6 +127,10 @@ export class SettingsViewModel extends ViewModel {
return this._keyBackupViewModel;
}
get featuresViewModel() {
return this._featuresViewModel;
}
get storageQuota() {
return this._formatBytes(this._estimate?.quota);
}
@ -150,8 +156,14 @@ export class SettingsViewModel extends ViewModel {
}
async exportLogs() {
const logExport = await this.logger.export();
this.platform.saveFileAs(logExport.asBlob(), `hydrogen-logs-${this.platform.clock.now()}.json`);
const logs = await this.exportLogsBlob();
this.platform.saveFileAs(logs, `hydrogen-logs-${this.platform.clock.now()}.json`);
}
async exportLogsBlob() {
const persister = this.logger.reporters.find(r => typeof r.export === "function");
const logExport = await persister.export();
return logExport.asBlob();
}
get canSendLogsToServer() {
@ -169,29 +181,13 @@ export class SettingsViewModel extends ViewModel {
}
async sendLogsToServer() {
const {bugReportEndpointUrl} = this.platform.config;
if (bugReportEndpointUrl) {
this._logsFeedbackMessage = this.i18n`Sending logs…`;
this._logsFeedbackMessage = this.i18n`Sending logs…`;
try {
await submitLogsFromSessionToDefaultServer(this._session, this.platform);
this._logsFeedbackMessage = this.i18n`Logs sent succesfully!`;
} catch (err) {
this._logsFeedbackMessage = err.message;
this.emitChange();
try {
const logExport = await this.logger.export();
await submitLogsToRageshakeServer(
{
app: "hydrogen",
userAgent: this.platform.description,
version: DEFINE_VERSION,
text: `Submit logs from settings for user ${this._session.userId} on device ${this._session.deviceId}`,
},
logExport.asBlob(),
bugReportEndpointUrl,
this.platform.request
);
this._logsFeedbackMessage = this.i18n`Logs sent succesfully!`;
this.emitChange();
} catch (err) {
this._logsFeedbackMessage = err.message;
this.emitChange();
}
}
}

35
src/domain/session/toast/BaseToastNotificationViewModel.ts Normal file
View File

@ -0,0 +1,35 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ErrorReportViewModel} from "../../ErrorReportViewModel";
import {Options as BaseOptions} from "../../ViewModel";
import type {Session} from "../../../matrix/Session.js";
import {SegmentType} from "../../navigation";
export type BaseClassOptions<N extends object = SegmentType> = {
dismiss: () => void;
session: Session;
} & BaseOptions<N>;
export abstract class BaseToastNotificationViewModel<N extends object = SegmentType, O extends BaseClassOptions<N> = BaseClassOptions<N>> extends ErrorReportViewModel<N, O> {
constructor(options: O) {
super(options);
}
dismiss(): void {
this.getOption("dismiss")();
}
}

92
src/domain/session/toast/CallToastNotificationViewModel.ts Normal file
View File

@ -0,0 +1,92 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {GroupCall} from "../../../matrix/calls/group/GroupCall";
import type {Room} from "../../../matrix/room/Room.js";
import {IAvatarContract, avatarInitials, getIdentifierColorNumber, getAvatarHttpUrl} from "../../avatar";
import {LocalMedia} from "../../../matrix/calls/LocalMedia";
import {BaseClassOptions, BaseToastNotificationViewModel} from "./BaseToastNotificationViewModel";
import {SegmentType} from "../../navigation";
type Options<N extends MinimumNeededSegmentType = SegmentType> = {
call: GroupCall;
room: Room;
} & BaseClassOptions<N>;
// Since we access the room segment below, the segment type
// needs to at least contain the room segment!
type MinimumNeededSegmentType = {
"room": string;
};
export class CallToastNotificationViewModel<N extends MinimumNeededSegmentType = SegmentType, O extends Options<N> = Options<N>> extends BaseToastNotificationViewModel<N, O> implements IAvatarContract {
constructor(options: O) {
super(options);
this.track(this.call.members.observeSize().subscribe(() => {
this.emitChange("memberCount");
}));
// Dismiss the toast if the room is opened manually
this.track(
this.navigation.observe("room").subscribe((roomId) => {
if ((roomId as unknown as string) === this.call.roomId) {
this.dismiss();
}
}));
}
async join(): Promise<void> {
await this.logAndCatch("CallToastNotificationViewModel.join", async (log) => {
const stream = await this.platform.mediaDevices.getMediaTracks(false, true);
const localMedia = new LocalMedia().withUserMedia(stream);
await this.call.join(localMedia, log);
const url = this.urlRouter.openRoomActionUrl(this.call.roomId);
this.urlRouter.pushUrl(url);
});
}
get call(): GroupCall {
return this.getOption("call");
}
private get room(): Room {
return this.getOption("room");
}
get roomName(): string {
return this.room.name;
}
get memberCount(): number {
return this.call.members.size;
}
get avatarLetter(): string {
return avatarInitials(this.roomName);
}
get avatarColorNumber(): number {
return getIdentifierColorNumber(this.room.avatarColorId);
}
avatarUrl(size: number): string | undefined {
return getAvatarHttpUrl(this.room.avatarUrl, size, this.platform, this.room.mediaRepository);
}
get avatarTitle(): string {
return this.roomName;
}
}

96
src/domain/session/toast/ToastCollectionViewModel.ts Normal file
View File

@ -0,0 +1,96 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {CallToastNotificationViewModel} from "./CallToastNotificationViewModel";
import {ObservableArray} from "../../../observable";
import {ViewModel, Options as BaseOptions} from "../../ViewModel";
import type {GroupCall} from "../../../matrix/calls/group/GroupCall";
import type {Room} from "../../../matrix/room/Room.js";
import type {Session} from "../../../matrix/Session.js";
import type {SegmentType} from "../../navigation";
import { RoomStatus } from "../../../lib";
type Options = {
session: Session;
} & BaseOptions;
export class ToastCollectionViewModel extends ViewModel<SegmentType, Options> {
public readonly toastViewModels: ObservableArray<CallToastNotificationViewModel> = new ObservableArray();
constructor(options: Options) {
super(options);
const session = this.getOption("session");
if (this.features.calls) {
const callsObservableMap = session.callHandler.calls;
this.track(callsObservableMap.subscribe(this));
}
}
async onAdd(_, call: GroupCall) {
if (this._shouldShowNotification(call)) {
const room = await this._findRoomForCall(call);
const dismiss = () => {
const idx = this.toastViewModels.array.findIndex(vm => vm.call === call);
if (idx !== -1) {
this.toastViewModels.remove(idx);
}
};
this.toastViewModels.append(
new CallToastNotificationViewModel(this.childOptions({ call, room, dismiss }))
);
}
}
onRemove(_, call: GroupCall) {
const idx = this.toastViewModels.array.findIndex(vm => vm.call === call);
if (idx !== -1) {
this.toastViewModels.remove(idx);
}
}
onUpdate(_, call: GroupCall) {
const idx = this.toastViewModels.array.findIndex(vm => vm.call === call);
if (idx !== -1) {
this.toastViewModels.update(idx, this.toastViewModels.at(idx)!);
}
}
onReset() {
for (let i = 0; i < this.toastViewModels.length; ++i) {
this.toastViewModels.remove(i);
}
}
private async _findRoomForCall(call: GroupCall): Promise<Room> {
const id = call.roomId;
const session = this.getOption("session");
const rooms = session.rooms;
// Make sure that we know of this room,
// otherwise wait for it to come through sync
const observable = await session.observeRoomStatus(id);
await observable.waitFor(s => s === RoomStatus.Joined).promise;
const room = rooms.get(id);
return room;
}
private _shouldShowNotification(call: GroupCall): boolean {
const currentlyOpenedRoomId = this.navigation.path.get("room")?.value;
if (!call.isLoadedFromStorage && call.roomId !== currentlyOpenedRoomId && !call.usesFoci) {
return true;
}
return false;
}
}

50
src/features.ts Normal file
View File

@ -0,0 +1,50 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {SettingsStorage} from "./platform/web/dom/SettingsStorage";
export enum FeatureFlag {
Calls = 1 << 0,
}
export class FeatureSet {
constructor(public readonly flags: number = 0) {}
withFeature(flag: FeatureFlag): FeatureSet {
return new FeatureSet(this.flags | flag);
}
withoutFeature(flag: FeatureFlag): FeatureSet {
return new FeatureSet(this.flags ^ flag);
}
isFeatureEnabled(flag: FeatureFlag): boolean {
return (this.flags & flag) !== 0;
}
get calls(): boolean {
return this.isFeatureEnabled(FeatureFlag.Calls);
}
static async load(settingsStorage: SettingsStorage): Promise<FeatureSet> {
const flags = await settingsStorage.getInt("enabled_features") || 0;
return new FeatureSet(flags);
}
async store(settingsStorage: SettingsStorage): Promise<void> {
await settingsStorage.setInt("enabled_features", this.flags);
}
}

16
src/lib.ts
View File

@ -14,9 +14,19 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
export {Logger} from "./logging/Logger";
export type {ILogItem} from "./logging/types";
export {IDBLogPersister} from "./logging/IDBLogPersister";
export {ConsoleReporter} from "./logging/ConsoleReporter";
export {Platform} from "./platform/web/Platform.js";
export {Client, LoadStatus} from "./matrix/Client.js";
export {RoomStatus} from "./matrix/room/common";
// export everything needed to observe state events on all rooms using session.observeRoomState
export type {RoomStateHandler} from "./matrix/room/state/types";
export type {MemberChange} from "./matrix/room/members/RoomMember";
export type {Transaction} from "./matrix/storage/idb/Transaction";
export type {Room} from "./matrix/room/Room";
export type {StateEvent} from "./matrix/storage/types";
export {PowerLevels} from "./matrix/room/PowerLevels.js";
// export main view & view models
export {createNavigation, createRouter} from "./domain/navigation/index";
@ -72,6 +82,7 @@ export {AvatarView} from "./platform/web/ui/AvatarView.js";
export {RoomType} from "./matrix/room/common";
export {EventEmitter} from "./utils/EventEmitter";
export {Disposables} from "./utils/Disposables";
export {LocalMedia} from "./matrix/calls/LocalMedia";
// these should eventually be moved to another library
export {
ObservableArray,
@ -79,9 +90,10 @@ export {
MappedList,
AsyncMappedList,
ConcatList,
} from "./observable";
ObservableMap
} from "./observable/index";
export {
BaseObservableValue,
ObservableValue,
RetainedObservableValue
} from "./observable/ObservableValue";
} from "./observable/value";

31
src/logging/ConsoleLogger.ts → src/logging/ConsoleReporter.ts
View File

@ -13,17 +13,28 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseLogger} from "./BaseLogger";
import {LogItem} from "./LogItem";
import type {ILogItem, LogItemValues, ILogExport} from "./types";
export class ConsoleLogger extends BaseLogger {
_persistItem(item: LogItem): void {
printToConsole(item);
import type {ILogger, ILogItem, LogItemValues, ILogReporter} from "./types";
import type {LogItem} from "./LogItem";
export class ConsoleReporter implements ILogReporter {
private logger?: ILogger;
reportItem(item: ILogItem): void {
printToConsole(item as LogItem);
}
async export(): Promise<ILogExport | undefined> {
return undefined;
setLogger(logger: ILogger) {
this.logger = logger;
}
printOpenItems(): void {
if (!this.logger) {
return;
}
for (const item of this.logger.getOpenRootItems()) {
this.reportItem(item);
}
}
}
@ -39,7 +50,7 @@ function filterValues(values: LogItemValues): LogItemValues | null {
}
function printToConsole(item: LogItem): void {
const label = `${itemCaption(item)} (${item.duration}ms)`;
const label = `${itemCaption(item)} (@${item.start}ms, duration: ${item.duration}ms)`;
const filteredValues = filterValues(item.values);
const shouldGroup = item.children || filteredValues;
if (shouldGroup) {
@ -78,6 +89,8 @@ function itemCaption(item: ILogItem): string {
return `${item.values.l} ${item.values.id}`;
} else if (item.values.l && typeof item.values.status !== "undefined") {
return `${item.values.l} (${item.values.status})`;
} else if (item.values.l && typeof item.values.type !== "undefined") {
return `${item.values.l} (${item.values.type})`;
} else if (item.values.l && item.error) {
return `${item.values.l} failed`;
} else if (typeof item.values.ref !== "undefined") {

154
src/logging/IDBLogger.ts → src/logging/IDBLogPersister.ts
View File

@ -22,36 +22,69 @@ import {
iterateCursor,
fetchResults,
} from "../matrix/storage/idb/utils";
import {BaseLogger} from "./BaseLogger";
import type {Interval} from "../platform/web/dom/Clock";
import type {Platform} from "../platform/web/Platform.js";
import type {BlobHandle} from "../platform/web/dom/BlobHandle.js";
import type {ILogItem, ILogExport, ISerializedItem} from "./types";
import type {LogFilter} from "./LogFilter";
import type {ILogItem, ILogger, ILogReporter, ISerializedItem} from "./types";
import {LogFilter} from "./LogFilter";
type QueuedItem = {
json: string;
id?: number;
}
export class IDBLogger extends BaseLogger {
private readonly _name: string;
private readonly _limit: number;
type Options = {
name: string,
flushInterval?: number,
limit?: number,
platform: Platform,
serializedTransformer?: (item: ISerializedItem) => ISerializedItem
}
export class IDBLogPersister implements ILogReporter {
private readonly _flushInterval: Interval;
private _queuedItems: QueuedItem[];
private readonly options: Options;
private logger?: ILogger;
constructor(options: {name: string, flushInterval?: number, limit?: number, platform: Platform, serializedTransformer?: (item: ISerializedItem) => ISerializedItem}) {
super(options);
const {name, flushInterval = 60 * 1000, limit = 3000} = options;
this._name = name;
this._limit = limit;
constructor(options: Options) {
this.options = options;
this._queuedItems = this._loadQueuedItems();
// TODO: also listen for unload just in case sync keeps on running after pagehide is fired?
window.addEventListener("pagehide", this, false);
this._flushInterval = this._platform.clock.createInterval(() => this._tryFlush(), flushInterval);
this._flushInterval = this.options.platform.clock.createInterval(
() => this._tryFlush(),
this.options.flushInterval ?? 60 * 1000
);
}
// TODO: move dispose to ILogger, listen to pagehide elsewhere and call dispose from there, which calls _finishAllAndFlush
setLogger(logger: ILogger): void {
this.logger = logger;
}
reportItem(logItem: ILogItem, filter: LogFilter, forced: boolean): void {
const queuedItem = this.prepareItemForQueue(logItem, filter, forced);
if (queuedItem) {
this._queuedItems.push(queuedItem);
}
}
async export(): Promise<IDBLogExport> {
const db = await this._openDB();
try {
const txn = db.transaction(["logs"], "readonly");
const logs = txn.objectStore("logs");
const storedItems: QueuedItem[] = await fetchResults(logs.openCursor(), () => false);
const openItems = this.getSerializedOpenItems();
const allItems = storedItems.concat(this._queuedItems).concat(openItems);
return new IDBLogExport(allItems, this, this.options.platform);
} finally {
try {
db.close();
} catch (e) {}
}
}
dispose(): void {
window.removeEventListener("pagehide", this, false);
this._flushInterval.dispose();
@ -63,7 +96,7 @@ export class IDBLogger extends BaseLogger {
}
}
async _tryFlush(): Promise<void> {
private async _tryFlush(): Promise<void> {
const db = await this._openDB();
try {
const txn = db.transaction(["logs"], "readwrite");
@ -73,9 +106,10 @@ export class IDBLogger extends BaseLogger {
logs.add(i);
}
const itemCount = await reqAsPromise(logs.count());
if (itemCount > this._limit) {
const limit = this.options.limit ?? 3000;
if (itemCount > limit) {
// delete an extra 10% so we don't need to delete every time we flush
let deleteAmount = (itemCount - this._limit) + Math.round(0.1 * this._limit);
let deleteAmount = (itemCount - limit) + Math.round(0.1 * limit);
await iterateCursor(logs.openCursor(), (_, __, cursor) => {
cursor.delete();
deleteAmount -= 1;
@ -93,14 +127,16 @@ export class IDBLogger extends BaseLogger {
}
}
_finishAllAndFlush(): void {
this._finishOpenItems();
this.log({l: "pagehide, closing logs", t: "navigation"});
private _finishAllAndFlush(): void {
if (this.logger) {
this.logger.log({l: "pagehide, closing logs", t: "navigation"});
this.logger.forceFinish();
}
this._persistQueuedItems(this._queuedItems);
}
_loadQueuedItems(): QueuedItem[] {
const key = `${this._name}_queuedItems`;
private _loadQueuedItems(): QueuedItem[] {
const key = `${this.options.name}_queuedItems`;
try {
const json = window.localStorage.getItem(key);
if (json) {
@ -113,44 +149,32 @@ export class IDBLogger extends BaseLogger {
return [];
}
_openDB(): Promise<IDBDatabase> {
return openDatabase(this._name, db => db.createObjectStore("logs", {keyPath: "id", autoIncrement: true}), 1);
private _openDB(): Promise<IDBDatabase> {
return openDatabase(this.options.name, db => db.createObjectStore("logs", {keyPath: "id", autoIncrement: true}), 1);
}
_persistItem(logItem: ILogItem, filter: LogFilter, forced: boolean): void {
const serializedItem = logItem.serialize(filter, undefined, forced);
private prepareItemForQueue(logItem: ILogItem, filter: LogFilter, forced: boolean): QueuedItem | undefined {
let serializedItem = logItem.serialize(filter, undefined, forced);
if (serializedItem) {
const transformedSerializedItem = this._serializedTransformer(serializedItem);
this._queuedItems.push({
json: JSON.stringify(transformedSerializedItem)
});
if (this.options.serializedTransformer) {
serializedItem = this.options.serializedTransformer(serializedItem);
}
return {
json: JSON.stringify(serializedItem)
};
}
}
_persistQueuedItems(items: QueuedItem[]): void {
private _persistQueuedItems(items: QueuedItem[]): void {
try {
window.localStorage.setItem(`${this._name}_queuedItems`, JSON.stringify(items));
window.localStorage.setItem(`${this.options.name}_queuedItems`, JSON.stringify(items));
} catch (e) {
console.error("Could not persist queued log items in localStorage, they will likely be lost", e);
}
}
async export(): Promise<ILogExport> {
const db = await this._openDB();
try {
const txn = db.transaction(["logs"], "readonly");
const logs = txn.objectStore("logs");
const storedItems: QueuedItem[] = await fetchResults(logs.openCursor(), () => false);
const allItems = storedItems.concat(this._queuedItems);
return new IDBLogExport(allItems, this, this._platform);
} finally {
try {
db.close();
} catch (e) {}
}
}
async _removeItems(items: QueuedItem[]): Promise<void> {
/** @internal called by ILogExport.removeFromStore */
async removeItems(items: QueuedItem[]): Promise<void> {
const db = await this._openDB();
try {
const txn = db.transaction(["logs"], "readwrite");
@ -173,14 +197,29 @@ export class IDBLogger extends BaseLogger {
} catch (e) {}
}
}
private getSerializedOpenItems(): QueuedItem[] {
const openItems: QueuedItem[] = [];
if (!this.logger) {
return openItems;
}
const filter = new LogFilter();
for(const item of this.logger!.getOpenRootItems()) {
const openItem = this.prepareItemForQueue(item, filter, false);
if (openItem) {
openItems.push(openItem);
}
}
return openItems;
}
}
class IDBLogExport implements ILogExport {
export class IDBLogExport {
private readonly _items: QueuedItem[];
private readonly _logger: IDBLogger;
private readonly _logger: IDBLogPersister;
private readonly _platform: Platform;
constructor(items: QueuedItem[], logger: IDBLogger, platform: Platform) {
constructor(items: QueuedItem[], logger: IDBLogPersister, platform: Platform) {
this._items = items;
this._logger = logger;
this._platform = platform;
@ -194,10 +233,17 @@ class IDBLogExport implements ILogExport {
* @return {Promise}
*/
removeFromStore(): Promise<void> {
return this._logger._removeItems(this._items);
return this._logger.removeItems(this._items);
}
asBlob(): BlobHandle {
const json = this.toJSON();
const buffer: Uint8Array = this._platform.encoding.utf8.encode(json);
const blob: BlobHandle = this._platform.createBlob(buffer, "application/json");
return blob;
}
toJSON(): string {
const log = {
formatVersion: 1,
appVersion: this._platform.updateService?.version,
@ -205,8 +251,6 @@ class IDBLogExport implements ILogExport {
items: this._items.map(i => JSON.parse(i.json))
};
const json = JSON.stringify(log);
const buffer: Uint8Array = this._platform.encoding.utf8.encode(json);
const blob: BlobHandle = this._platform.createBlob(buffer, "application/json");
return blob;
return json;
}
}

17
src/logging/LogItem.ts
View File

@ -16,7 +16,7 @@ limitations under the License.
*/
import {LogLevel, LogFilter} from "./LogFilter";
import type {BaseLogger} from "./BaseLogger";
import type {Logger} from "./Logger";
import type {ISerializedItem, ILogItem, LogItemValues, LabelOrValues, FilterCreator, LogCallback} from "./types";
export class LogItem implements ILogItem {
@ -25,11 +25,11 @@ export class LogItem implements ILogItem {
public error?: Error;
public end?: number;
private _values: LogItemValues;
private _logger: BaseLogger;
protected _logger: Logger;
private _filterCreator?: FilterCreator;
private _children?: Array<LogItem>;
constructor(labelOrValues: LabelOrValues, logLevel: LogLevel, logger: BaseLogger, filterCreator?: FilterCreator) {
constructor(labelOrValues: LabelOrValues, logLevel: LogLevel, logger: Logger, filterCreator?: FilterCreator) {
this._logger = logger;
this.start = logger._now();
// (l)abel
@ -38,7 +38,7 @@ export class LogItem implements ILogItem {
this._filterCreator = filterCreator;
}
/** start a new root log item and run it detached mode, see BaseLogger.runDetached */
/** start a new root log item and run it detached mode, see Logger.runDetached */
runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem {
return this._logger.runDetached(labelOrValues, callback, logLevel, filterCreator);
}
@ -221,6 +221,11 @@ export class LogItem implements ILogItem {
}
}
/** @internal */
forceFinish(): void {
this.finish();
}
// expose log level without needing import everywhere
get level(): typeof LogLevel {
return LogLevel;
@ -235,7 +240,7 @@ export class LogItem implements ILogItem {
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): LogItem {
if (this.end) {
console.trace("log item is finished, additional logs will likely not be recorded");
console.trace(`log item ${this.values.l} finished, additional log ${JSON.stringify(labelOrValues)} will likely not be recorded`);
}
if (!logLevel) {
logLevel = this.logLevel || LogLevel.Info;
@ -248,7 +253,7 @@ export class LogItem implements ILogItem {
return item;
}
get logger(): BaseLogger {
get logger(): Logger {
return this._logger;
}

66
src/logging/BaseLogger.ts → src/logging/Logger.ts
View File

@ -17,23 +17,33 @@ limitations under the License.
import {LogItem} from "./LogItem";
import {LogLevel, LogFilter} from "./LogFilter";
import type {ILogger, ILogExport, FilterCreator, LabelOrValues, LogCallback, ILogItem, ISerializedItem} from "./types";
import type {ILogger, ILogReporter, FilterCreator, LabelOrValues, LogCallback, ILogItem, ISerializedItem} from "./types";
import type {Platform} from "../platform/web/Platform.js";
export abstract class BaseLogger implements ILogger {
export class Logger implements ILogger {
protected _openItems: Set<LogItem> = new Set();
protected _platform: Platform;
protected _serializedTransformer: (item: ISerializedItem) => ISerializedItem;
public readonly reporters: ILogReporter[] = [];
constructor({platform, serializedTransformer = (item: ISerializedItem) => item}) {
constructor({platform}) {
this._platform = platform;
this._serializedTransformer = serializedTransformer;
}
log(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info): void {
log(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info): ILogItem {
const item = new LogItem(labelOrValues, logLevel, this);
item.end = item.start;
this._persistItem(item, undefined, false);
return item;
}
/** Prefer `run()` or `log()` above this method; only use it if you have a long-running operation
* *without* a single call stack that should be logged into one sub-tree.
* You need to call `finish()` on the returned item or it will stay open until the app unloads. */
child(labelOrValues: LabelOrValues, logLevel: LogLevel = LogLevel.Info, filterCreator?: FilterCreator): ILogItem {
const item = new DeferredPersistRootLogItem(labelOrValues, logLevel, this, filterCreator);
this._openItems.add(item);
return item;
}
/** if item is a log item, wrap the callback in a child of it, otherwise start a new root log item. */
@ -70,10 +80,10 @@ export abstract class BaseLogger implements ILogger {
return this._run(item, callback, logLevel, true, filterCreator);
}
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: true, filterCreator?: FilterCreator): T;
private _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: true, filterCreator?: FilterCreator): T;
// we don't return if we don't throw, as we don't have anything to return when an error is caught but swallowed for the fire-and-forget case.
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: false, filterCreator?: FilterCreator): void;
_run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: boolean, filterCreator?: FilterCreator): T | void {
private _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: false, filterCreator?: FilterCreator): void;
private _run<T>(item: LogItem, callback: LogCallback<T>, logLevel: LogLevel, wantResult: boolean, filterCreator?: FilterCreator): T | void {
this._openItems.add(item);
const finishItem = () => {
@ -125,9 +135,18 @@ export abstract class BaseLogger implements ILogger {
}
}
_finishOpenItems() {
addReporter(reporter: ILogReporter): void {
reporter.setLogger(this);
this.reporters.push(reporter);
}
getOpenRootItems(): Iterable<ILogItem> {
return this._openItems;
}
forceFinish() {
for (const openItem of this._openItems) {
openItem.finish();
openItem.forceFinish();
try {
// for now, serialize with an all-permitting filter
// as the createFilter function would get a distorted image anyway
@ -141,20 +160,43 @@ export abstract class BaseLogger implements ILogger {
this._openItems.clear();
}
abstract _persistItem(item: LogItem, filter?: LogFilter, forced?: boolean): void;
/** @internal */
_removeItemFromOpenList(item: LogItem): void {
this._openItems.delete(item);
}
abstract export(): Promise<ILogExport | undefined>;
/** @internal */
_persistItem(item: LogItem, filter?: LogFilter, forced?: boolean): void {
for (var i = 0; i < this.reporters.length; i += 1) {
this.reporters[i].reportItem(item, filter, forced);
}
}
// expose log level without needing
get level(): typeof LogLevel {
return LogLevel;
}
/** @internal */
_now(): number {
return this._platform.clock.now();
}
/** @internal */
_createRefId(): number {
return Math.round(this._platform.random() * Number.MAX_SAFE_INTEGER);
}
}
class DeferredPersistRootLogItem extends LogItem {
finish() {
super.finish();
(this._logger as Logger)._persistItem(this, undefined, false);
(this._logger as Logger)._removeItemFromOpenList(this);
}
forceFinish() {
super.finish();
/// no need to persist when force-finishing as _finishOpenItems above will do it
}
}

39
src/logging/NullLogger.ts
View File

@ -14,14 +14,32 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {LogLevel} from "./LogFilter";
import type {ILogger, ILogExport, ILogItem, LabelOrValues, LogCallback, LogItemValues} from "./types";
import type {ILogger, ILogItem, ILogReporter, LabelOrValues, LogCallback, LogItemValues} from "./types";
function noop (): void {}
export class NullLogger implements ILogger {
public readonly item: ILogItem = new NullLogItem(this);
log(): void {}
log(labelOrValues: LabelOrValues): ILogItem {
return this.item;
}
addReporter() {}
get reporters(): ReadonlyArray<ILogReporter> {
return [];
}
getOpenRootItems(): Iterable<ILogItem> {
return [];
}
forceFinish(): void {}
child(labelOrValues: LabelOrValues): ILogItem {
return this.item;
}
run<T>(_, callback: LogCallback<T>): T {
return callback(this.item);
@ -39,11 +57,7 @@ export class NullLogger implements ILogger {
new Promise(r => r(callback(this.item))).then(noop, noop);
return this.item;
}
async export(): Promise<ILogExport | undefined> {
return undefined;
}
get level(): typeof LogLevel {
return LogLevel;
}
@ -61,13 +75,19 @@ export class NullLogItem implements ILogItem {
}
wrap<T>(_: LabelOrValues, callback: LogCallback<T>): T {
return this.run(callback);
}
run<T>(callback: LogCallback<T>): T {
return callback(this);
}
log(): ILogItem {
log(labelOrValues: LabelOrValues): ILogItem {
return this;
}
set(): ILogItem { return this; }
set(labelOrValues: LabelOrValues): ILogItem { return this; }
runDetached(_: LabelOrValues, callback: LogCallback<unknown>): ILogItem {
new Promise(r => r(callback(this))).then(noop, noop);
@ -99,6 +119,7 @@ export class NullLogItem implements ILogItem {
}
finish(): void {}
forceFinish(): void {}
serialize(): undefined {
return undefined;

36
src/logging/types.ts
View File

@ -16,7 +16,6 @@ limitations under the License.
*/
import {LogLevel, LogFilter} from "./LogFilter";
import type {BaseLogger} from "./BaseLogger";
import type {BlobHandle} from "../platform/web/dom/BlobHandle.js";
export interface ISerializedItem {
@ -40,8 +39,10 @@ export interface ILogItem {
readonly level: typeof LogLevel;
readonly end?: number;
readonly start?: number;
readonly values: LogItemValues;
readonly values: Readonly<LogItemValues>;
wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
/*** This is sort of low-level, you probably want to use wrap. If you do use it, it should only be called once. */
run<T>(callback: LogCallback<T>): T;
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem;
set(key: string | object, value: unknown): ILogItem;
runDetached(labelOrValues: LabelOrValues, callback: LogCallback<unknown>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
@ -51,22 +52,41 @@ export interface ILogItem {
catch(err: Error): Error;
serialize(filter: LogFilter, parentStartTime: number | undefined, forced: boolean): ISerializedItem | undefined;
finish(): void;
forceFinish(): void;
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
}
/*
extend both ILogger and ILogItem from this interface, but need to rename ILogger.run => wrap then. Or both to `span`?
export interface ILogItemCreator {
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
refDetached(logItem: ILogItem, logLevel?: LogLevel): void;
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem;
wrap<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
get level(): typeof LogLevel;
}
*/
export interface ILogger {
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): void;
log(labelOrValues: LabelOrValues, logLevel?: LogLevel): ILogItem;
child(labelOrValues: LabelOrValues, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
wrapOrRun<T>(item: ILogItem | undefined, labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
runDetached<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): ILogItem;
run<T>(labelOrValues: LabelOrValues, callback: LogCallback<T>, logLevel?: LogLevel, filterCreator?: FilterCreator): T;
export(): Promise<ILogExport | undefined>;
get level(): typeof LogLevel;
getOpenRootItems(): Iterable<ILogItem>;
addReporter(reporter: ILogReporter): void;
get reporters(): ReadonlyArray<ILogReporter>;
/**
* force-finishes any open items and passes them to the reporter, with the forced flag set.
* Good think to do when the page is being closed to not lose any logs.
**/
forceFinish(): void;
}
export interface ILogExport {
get count(): number;
removeFromStore(): Promise<void>;
asBlob(): BlobHandle;
export interface ILogReporter {
setLogger(logger: ILogger): void;
reportItem(item: ILogItem, filter?: LogFilter, forced?: boolean): void;
}
export type LogItemValues = {

7
src/matrix/Client.js
View File

@ -18,7 +18,7 @@ limitations under the License.
import {createEnum} from "../utils/enum";
import {lookupHomeserver} from "./well-known.js";
import {AbortableOperation} from "../utils/AbortableOperation";
import {ObservableValue} from "../observable/ObservableValue";
import {ObservableValue} from "../observable/value";
import {HomeServerApi} from "./net/HomeServerApi";
import {Reconnector, ConnectionStatus} from "./net/Reconnector";
import {ExponentialRetryDelay} from "./net/ExponentialRetryDelay";
@ -31,6 +31,7 @@ import {TokenLoginMethod} from "./login/TokenLoginMethod";
import {SSOLoginHelper} from "./login/SSOLoginHelper";
import {getDehydratedDevice} from "./e2ee/Dehydration.js";
import {Registration} from "./registration/Registration";
import {FeatureSet} from "../features";
export const LoadStatus = createEnum(
"NotLoading",
@ -53,7 +54,7 @@ export const LoginFailure = createEnum(
);
export class Client {
constructor(platform) {
constructor(platform, features = new FeatureSet(0)) {
this._platform = platform;
this._sessionStartedByReconnector = false;
this._status = new ObservableValue(LoadStatus.NotLoading);
@ -68,6 +69,7 @@ export class Client {
this._olmPromise = platform.loadOlm();
this._workerPromise = platform.loadOlmWorker();
this._accountSetup = undefined;
this._features = features;
}
createNewSessionId() {
@ -278,6 +280,7 @@ export class Client {
olmWorker,
mediaRepository,
platform: this._platform,
features: this._features
});
await this._session.load(log);
if (dehydratedDevice) {

45
src/matrix/DeviceMessageHandler.js
View File

@ -16,12 +16,15 @@ limitations under the License.
import {OLM_ALGORITHM} from "./e2ee/common.js";
import {countBy, groupBy} from "../utils/groupBy";
import {LRUCache} from "../utils/LRUCache";
export class DeviceMessageHandler {
constructor({storage}) {
constructor({storage, callHandler}) {
this._storage = storage;
this._olmDecryption = null;
this._megolmDecryption = null;
this._callHandler = callHandler;
this._senderDeviceCache = new LRUCache(10, di => di.curve25519Key);
}
enableEncryption({olmDecryption, megolmDecryption}) {
@ -49,6 +52,11 @@ export class DeviceMessageHandler {
log.child("decrypt_error").catch(err);
}
const newRoomKeys = this._megolmDecryption.roomKeysFromDeviceMessages(olmDecryptChanges.results, log);
// TODO: somehow include rooms that received a call to_device message in the sync state?
// or have updates flow through event emitter?
// well, we don't really need to update the room other then when a call starts or stops
// any changes within the call will be emitted on the call object?
return new SyncPreparation(olmDecryptChanges, newRoomKeys);
}
}
@ -58,7 +66,40 @@ export class DeviceMessageHandler {
// write olm changes
prep.olmDecryptChanges.write(txn);
const didWriteValues = await Promise.all(prep.newRoomKeys.map(key => this._megolmDecryption.writeRoomKey(key, txn)));
return didWriteValues.some(didWrite => !!didWrite);
const hasNewRoomKeys = didWriteValues.some(didWrite => !!didWrite);
return {
hasNewRoomKeys,
decryptionResults: prep.olmDecryptChanges.results
};
}
async afterSyncCompleted(decryptionResults, deviceTracker, hsApi, log) {
if (this._callHandler) {
// if we don't have a device, we need to fetch the device keys the message claims
// and check the keys, and we should only do network requests during
// sync processing in the afterSyncCompleted step.
const callMessages = decryptionResults.filter(dr => this._callHandler.handlesDeviceMessageEventType(dr.event?.type));
if (callMessages.length) {
await log.wrap("process call signalling messages", async log => {
for (const dr of callMessages) {
// serialize device loading, so subsequent messages for the same device take advantage of the cache
const device = await deviceTracker.deviceForId(dr.event.sender, dr.event.content.device_id, hsApi, log);
dr.setDevice(device);
if (dr.isVerified) {
this._callHandler.handleDeviceMessage(dr.event, dr.userId, dr.deviceId, log);
} else {
log.log({
l: "could not verify olm fingerprint key matches, ignoring",
ed25519Key: dr.device.ed25519Key,
claimedEd25519Key: dr.claimedEd25519Key,
deviceId: device.deviceId,
userId: device.userId,
});
}
}
});
}
}
}
}

98
src/matrix/Session.js
View File

@ -45,14 +45,16 @@ import {
keyFromDehydratedDeviceKey as createSSSSKeyFromDehydratedDeviceKey
} from "./ssss/index";
import {SecretStorage} from "./ssss/SecretStorage";
import {ObservableValue, RetainedObservableValue} from "../observable/ObservableValue";
import {ObservableValue, RetainedObservableValue} from "../observable/value";
import {CallHandler} from "./calls/CallHandler";
import {RoomStateHandlerSet} from "./room/state/RoomStateHandlerSet";
const PICKLE_KEY = "DEFAULT_KEY";
const PUSHER_KEY = "pusher";
export class Session {
// sessionInfo contains deviceId, userId and homeserver
constructor({storage, hsApi, sessionInfo, olm, olmWorker, platform, mediaRepository}) {
constructor({storage, hsApi, sessionInfo, olm, olmWorker, platform, mediaRepository, features}) {
this._platform = platform;
this._storage = storage;
this._hsApi = hsApi;
@ -73,7 +75,8 @@ export class Session {
};
this._roomsBeingCreated = new ObservableMap();
this._user = new User(sessionInfo.userId);
this._deviceMessageHandler = new DeviceMessageHandler({storage});
this._roomStateHandler = new RoomStateHandlerSet();
this._deviceMessageHandler = new DeviceMessageHandler({storage, callHandler: this._callHandler});
this._olm = olm;
this._olmUtil = null;
this._e2eeAccount = null;
@ -100,6 +103,10 @@ export class Session {
this._createRoomEncryption = this._createRoomEncryption.bind(this);
this._forgetArchivedRoom = this._forgetArchivedRoom.bind(this);
this.needsKeyBackup = new ObservableValue(false);
if (features.calls) {
this._setupCallHandler();
}
}
get fingerprintKey() {
@ -118,6 +125,42 @@ export class Session {
return this._sessionInfo.userId;
}
get callHandler() {
return this._callHandler;
}
_setupCallHandler() {
this._callHandler = new CallHandler({
clock: this._platform.clock,
random: this._platform.random,
hsApi: this._hsApi,
encryptDeviceMessage: async (roomId, userId, deviceId, message, log) => {
if (!this._deviceTracker || !this._olmEncryption) {
log.set("encryption_disabled", true);
return;
}
const device = await log.wrap("get device key", async log => {
const device = this._deviceTracker.deviceForId(userId, deviceId, this._hsApi, log);
if (!device) {
log.set("not_found", true);
}
return device;
});
if (device) {
const encryptedMessages = await this._olmEncryption.encrypt(message.type, message.content, [device], this._hsApi, log);
return encryptedMessages;
}
},
storage: this._storage,
webRTC: this._platform.webRTC,
ownDeviceId: this._sessionInfo.deviceId,
ownUserId: this._sessionInfo.userId,
logger: this._platform.logger,
forceTURN: false,
});
this.observeRoomState(this._callHandler);
}
// called once this._e2eeAccount is assigned
_setupEncryption() {
// TODO: this should all go in a wrapper in e2ee/ that is bootstrapped by passing in the account
@ -452,6 +495,8 @@ export class Session {
this._megolmDecryption = undefined;
this._e2eeAccount?.dispose();
this._e2eeAccount = undefined;
this._callHandler?.dispose();
this._callHandler = undefined;
for (const room of this._rooms.values()) {
room.dispose();
}
@ -562,7 +607,8 @@ export class Session {
pendingEvents,
user: this._user,
createRoomEncryption: this._createRoomEncryption,
platform: this._platform
platform: this._platform,
roomStateHandler: this._roomStateHandler
});
}
@ -649,7 +695,9 @@ export class Session {
async writeSync(syncResponse, syncFilterId, preparation, txn, log) {
const changes = {
syncInfo: null,
e2eeAccountChanges: null
e2eeAccountChanges: null,
hasNewRoomKeys: false,
deviceMessageDecryptionResults: null,
};
const syncToken = syncResponse.next_batch;
if (syncToken !== this.syncToken) {
@ -670,7 +718,9 @@ export class Session {
}
if (preparation) {
changes.hasNewRoomKeys = await log.wrap("deviceMsgs", log => this._deviceMessageHandler.writeSync(preparation, txn, log));
const {hasNewRoomKeys, decryptionResults} = await log.wrap("deviceMsgs", log => this._deviceMessageHandler.writeSync(preparation, txn, log));
changes.hasNewRoomKeys = hasNewRoomKeys;
changes.deviceMessageDecryptionResults = decryptionResults;
}
// store account data
@ -711,6 +761,9 @@ export class Session {
if (changes.hasNewRoomKeys) {
this._keyBackup.get()?.flush(log);
}
if (changes.deviceMessageDecryptionResults) {
await this._deviceMessageHandler.afterSyncCompleted(changes.deviceMessageDecryptionResults, this._deviceTracker, this._hsApi, log);
}
}
_tryReplaceRoomBeingCreated(roomId, log) {
@ -730,7 +783,7 @@ export class Session {
}
}
applyRoomCollectionChangesAfterSync(inviteStates, roomStates, archivedRoomStates, log) {
async applyRoomCollectionChangesAfterSync(inviteStates, roomStates, archivedRoomStates, log) {
// update the collections after sync
for (const rs of roomStates) {
if (rs.shouldAdd) {
@ -894,16 +947,28 @@ export class Session {
async observeRoomStatus(roomId) {
let observable = this._observedRoomStatus.get(roomId);
if (!observable) {
const status = await this.getRoomStatus(roomId);
let status = undefined;
// Create and set the observable with value = undefined, so that
// we don't loose any sync changes that come in while we are busy
// calculating the current room status.
observable = new RetainedObservableValue(status, () => {
this._observedRoomStatus.delete(roomId);
});
this._observedRoomStatus.set(roomId, observable);
status = await this.getRoomStatus(roomId);
// If observable.value is not undefined anymore, then some
// change has come through the sync.
if (observable.get() === undefined) {
observable.set(status);
}
}
return observable;
}
observeRoomState(roomStateHandler) {
return this._roomStateHandler.subscribe(roomStateHandler);
}
/**
Creates an empty (summary isn't loaded) the archived room if it isn't
loaded already, assuming sync will either remove it (when rejoining) or
@ -950,6 +1015,7 @@ export class Session {
}
}
import {FeatureSet} from "../features";
export function tests() {
function createStorageMock(session, pendingEvents = []) {
return {
@ -983,9 +1049,19 @@ export function tests() {
return {
"session data is not modified until after sync": async (assert) => {
const session = new Session({storage: createStorageMock({
const storage = createStorageMock({
sync: {token: "a", filterId: 5}
}), sessionInfo: {userId: ""}});
});
const session = new Session({
storage,
sessionInfo: {userId: ""},
platform: {
clock: {
createTimeout: () => undefined
}
},
features: new FeatureSet(0)
});
await session.load();
let syncSet = false;
const syncTxn = {

4
src/matrix/Sync.js
View File

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableValue} from "../observable/ObservableValue";
import {ObservableValue} from "../observable/value";
import {createEnum} from "../utils/enum";
const INCREMENTAL_TIMEOUT = 30000;
@ -218,6 +218,7 @@ export class Sync {
_openPrepareSyncTxn() {
const storeNames = this._storage.storeNames;
return this._storage.readTxn([
storeNames.deviceIdentities, // to read device from olm messages
storeNames.olmSessions,
storeNames.inboundGroupSessions,
// to read fragments when loading sync writer when rejoining archived room
@ -337,6 +338,7 @@ export class Sync {
// to decrypt and store new room keys
storeNames.olmSessions,
storeNames.inboundGroupSessions,
storeNames.calls,
]);
}

289
src/matrix/calls/CallHandler.ts Normal file
View File

@ -0,0 +1,289 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableMap} from "../../observable/map";
import {WebRTC, PeerConnection} from "../../platform/types/WebRTC";
import {MediaDevices, Track} from "../../platform/types/MediaDevices";
import {handlesEventType} from "./PeerCall";
import {EventType, CallIntent, CallType} from "./callEventTypes";
import {GroupCall} from "./group/GroupCall";
import {makeId} from "../common";
import {CALL_LOG_TYPE} from "./common";
import {EVENT_TYPE as MEMBER_EVENT_TYPE, RoomMember} from "../room/members/RoomMember";
import {TurnServerSource} from "./TurnServerSource";
import type {LocalMedia} from "./LocalMedia";
import type {Room} from "../room/Room";
import type {MemberChange} from "../room/members/RoomMember";
import type {StateEvent} from "../storage/types";
import type {ILogItem, ILogger} from "../../logging/types";
import type {Platform} from "../../platform/web/Platform";
import type {BaseObservableMap} from "../../observable/map";
import type {SignallingMessage, MGroupCallBase} from "./callEventTypes";
import type {Options as GroupCallOptions} from "./group/GroupCall";
import type {Transaction} from "../storage/idb/Transaction";
import type {CallEntry} from "../storage/idb/stores/CallStore";
import type {Clock} from "../../platform/web/dom/Clock";
import type {RoomStateHandler} from "../room/state/types";
import type {MemberSync} from "../room/timeline/persistence/MemberWriter";
export type Options = Omit<GroupCallOptions, "emitUpdate" | "createTimeout" | "turnServerSource"> & {
clock: Clock
};
function getRoomMemberKey(roomId: string, userId: string): string {
return JSON.stringify(roomId)+`,`+JSON.stringify(userId);
}
export class CallHandler implements RoomStateHandler {
// group calls by call id
private readonly _calls: ObservableMap<string, GroupCall> = new ObservableMap<string, GroupCall>();
// map of `"roomId","userId"` to set of conf_id's they are in
private roomMemberToCallIds: Map<string, Set<string>> = new Map();
private groupCallOptions: GroupCallOptions;
private sessionId = makeId("s");
constructor(private readonly options: Options) {
this.groupCallOptions = Object.assign({}, this.options, {
turnServerSource: new TurnServerSource(this.options.hsApi, this.options.clock),
emitUpdate: (groupCall, params) => this._calls.update(groupCall.id, params),
createTimeout: this.options.clock.createTimeout,
sessionId: this.sessionId
});
}
loadCalls(intent?: CallIntent, log?: ILogItem) {
return this.options.logger.wrapOrRun(log, "CallHandler.loadCalls", async log => {
if (!intent) {
intent = CallIntent.Ring;
}
log.set("intent", intent);
const txn = await this._getLoadTxn();
const callEntries = await txn.calls.getByIntent(intent);
await this._loadCallEntries(callEntries, txn, log);
});
}
loadCallsForRoom(intent: CallIntent, roomId: string, log?: ILogItem) {
return this.options.logger.wrapOrRun(log, "CallHandler.loadCallsForRoom", async log => {
log.set("intent", intent);
log.set("roomId", roomId);
const txn = await this._getLoadTxn();
const callEntries = await txn.calls.getByIntentAndRoom(intent, roomId);
await this._loadCallEntries(callEntries, txn, log);
});
}
private async _getLoadTxn(): Promise<Transaction> {
const names = this.options.storage.storeNames;
const txn = await this.options.storage.readTxn([
names.calls,
names.roomState,
]);
return txn;
}
private async _loadCallEntries(callEntries: CallEntry[], txn: Transaction, log: ILogItem): Promise<void> {
log.set("entries", callEntries.length);
await Promise.all(callEntries.map(async callEntry => {
if (this._calls.get(callEntry.callId)) {
return;
}
const event = await txn.roomState.get(callEntry.roomId, EventType.GroupCall, callEntry.callId);
if (event) {
const call = new GroupCall(
event.event.state_key, // id
true, // isLoadedFromStorage
false, // newCall
callEntry.timestamp, // startTime
event.event.content, // callContent
event.roomId, // roomId
this.groupCallOptions // options
);
this._calls.set(call.id, call);
}
}));
const roomIds = Array.from(new Set(callEntries.map(e => e.roomId)));
await Promise.all(roomIds.map(async roomId => {
// TODO: don't load all members until we need them
const callsMemberEvents = await txn.roomState.getAllForType(roomId, EventType.GroupCallMember);
await Promise.all(callsMemberEvents.map(async entry => {
const userId = entry.event.sender;
const roomMemberState = await txn.roomState.get(roomId, MEMBER_EVENT_TYPE, userId);
let roomMember;
if (roomMemberState) {
roomMember = RoomMember.fromMemberEvent(roomMemberState.event);
}
if (!roomMember) {
// we'll be missing the member here if we received a call and it's members
// as pre-gap state and the members weren't active in the timeline we got.
roomMember = RoomMember.fromUserId(roomId, userId, "join");
}
this.handleCallMemberEvent(entry.event, roomMember, roomId, log);
}));
}));
log.set("newSize", this._calls.size);
}
createCall(roomId: string, type: CallType, name: string, intent?: CallIntent, log?: ILogItem): Promise<GroupCall> {
return this.options.logger.wrapOrRun(log, "CallHandler.createCall", async log => {
if (!intent) {
intent = CallIntent.Ring;
}
const call = new GroupCall(
makeId("conf-"), // id
false, // isLoadedFromStorage
true, // newCall
undefined, // startTime
{"m.name": name, "m.intent": intent}, // callContent
roomId, // roomId
this.groupCallOptions // options
);
this._calls.set(call.id, call);
try {
await call.create(type, log);
// store call info so it will ring again when reopening the app
const txn = await this.options.storage.readWriteTxn([this.options.storage.storeNames.calls]);
txn.calls.add({
intent: call.intent,
callId: call.id,
timestamp: this.options.clock.now(),
roomId: roomId
});
await txn.complete();
} catch (err) {
//if (err.name === "ConnectionError") {
// if we're offline, give up and remove the call again
this._calls.remove(call.id);
//}
throw err;
}
return call;
});
}
get calls(): BaseObservableMap<string, GroupCall> { return this._calls; }
// TODO: check and poll turn server credentials here
/** @internal */
async handleRoomState(room: Room, event: StateEvent, memberSync: MemberSync, txn: Transaction, log: ILogItem) {
if (event.type === EventType.GroupCall) {
this.handleCallEvent(event, room.id, txn, log);
}
if (event.type === EventType.GroupCallMember) {
let member = await memberSync.lookupMemberAtEvent(event.sender, event, txn);
if (!member) {
// we'll be missing the member here if we received a call and it's members
// as pre-gap state and the members weren't active in the timeline we got.
member = RoomMember.fromUserId(room.id, event.sender, "join");
}
this.handleCallMemberEvent(event, member, room.id, log);
}
}
/** @internal */
updateRoomMembers(room: Room, memberChanges: Map<string, MemberChange>) {
// TODO: also have map for roomId to calls, so we can easily update members
// we will also need this to get the call for a room
for (const call of this._calls.values()) {
if (call.roomId === room.id) {
call.updateRoomMembers(memberChanges);
}
}
}
/** @internal */
handlesDeviceMessageEventType(eventType: string): boolean {
return handlesEventType(eventType);
}
/** @internal */
handleDeviceMessage(message: SignallingMessage<MGroupCallBase>, userId: string, deviceId: string, log: ILogItem) {
// TODO: buffer messages for calls we haven't received the state event for yet?
const call = this._calls.get(message.content.conf_id);
call?.handleDeviceMessage(message, userId, deviceId, log);
}
private handleCallEvent(event: StateEvent, roomId: string, txn: Transaction, log: ILogItem) {
const callId = event.state_key;
let call = this._calls.get(callId);
if (call) {
call.updateCallEvent(event, log);
if (call.isTerminated) {
call.disconnect(log);
this._calls.remove(call.id);
txn.calls.remove(call.intent, roomId, call.id);
}
} else if(!event.content["m.terminated"]) {
// We don't have this call already and it isn't terminated, so create the call:
call = new GroupCall(
event.state_key, // id
false, // isLoadedFromStorage
false, // newCall
event.origin_server_ts, // startTime
event.content, // callContent
roomId, // roomId
this.groupCallOptions // options
);
this._calls.set(call.id, call);
txn.calls.add({
intent: call.intent,
callId: call.id,
timestamp: event.origin_server_ts,
roomId: roomId
});
}
}
private handleCallMemberEvent(event: StateEvent, member: RoomMember, roomId: string, log: ILogItem) {
const userId = event.state_key;
const roomMemberKey = getRoomMemberKey(roomId, userId)
const calls = event.content["m.calls"] ?? [];
for (const call of calls) {
const callId = call["m.call_id"];
const groupCall = this._calls.get(callId);
// TODO: also check the member when receiving the m.call event
groupCall?.updateMembership(userId, member, call, log);
};
const newCallIdsMemberOf = new Set<string>(calls.map(call => call["m.call_id"]));
let previousCallIdsMemberOf = this.roomMemberToCallIds.get(roomMemberKey);
// remove user as member of any calls not present anymore
if (previousCallIdsMemberOf) {
for (const previousCallId of previousCallIdsMemberOf) {
if (!newCallIdsMemberOf.has(previousCallId)) {
const groupCall = this._calls.get(previousCallId);
groupCall?.removeMembership(userId, log);
}
}
}
if (newCallIdsMemberOf.size === 0) {
this.roomMemberToCallIds.delete(roomMemberKey);
} else {
this.roomMemberToCallIds.set(roomMemberKey, newCallIdsMemberOf);
}
}
dispose() {
this.groupCallOptions.turnServerSource.dispose();
for(const call of this._calls.values()) {
call.dispose();
}
}
}

85
src/matrix/calls/LocalMedia.ts Normal file
View File

@ -0,0 +1,85 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {SDPStreamMetadataPurpose} from "./callEventTypes";
import {Stream} from "../../platform/types/MediaDevices";
import {SDPStreamMetadata} from "./callEventTypes";
import {getStreamVideoTrack, getStreamAudioTrack} from "./common";
export class LocalMedia {
constructor(
public readonly userMedia?: Stream,
public readonly screenShare?: Stream,
public readonly dataChannelOptions?: RTCDataChannelInit,
) {}
withUserMedia(stream: Stream) {
return new LocalMedia(stream, this.screenShare?.clone(), this.dataChannelOptions);
}
withScreenShare(stream: Stream) {
return new LocalMedia(this.userMedia?.clone(), stream, this.dataChannelOptions);
}
withDataChannel(options: RTCDataChannelInit): LocalMedia {
return new LocalMedia(this.userMedia?.clone(), this.screenShare?.clone(), options);
}
/**
* Create an instance of LocalMedia without audio track (for user preview)
*/
asPreview(): LocalMedia {
const media = this.clone();
const userMedia = media.userMedia;
if (userMedia && userMedia.getVideoTracks().length > 0) {
const audioTrack = getStreamAudioTrack(userMedia);
if (audioTrack) {
audioTrack.stop();
userMedia.removeTrack(audioTrack);
}
}
return media;
}
/** @internal */
replaceClone(oldClone: LocalMedia | undefined, oldOriginal: LocalMedia | undefined): LocalMedia {
const cloneOrAdoptStream = (oldOriginalStream: Stream | undefined, oldCloneStream: Stream | undefined, newStream: Stream | undefined): Stream | undefined => {
let stream;
if (oldOriginalStream?.id === newStream?.id) {
return oldCloneStream;
} else {
return newStream?.clone();
}
}
return new LocalMedia(
cloneOrAdoptStream(oldOriginal?.userMedia, oldClone?.userMedia, this.userMedia),
cloneOrAdoptStream(oldOriginal?.screenShare, oldClone?.screenShare, this.screenShare),
this.dataChannelOptions
);
}
/** @internal */
clone(): LocalMedia {
return new LocalMedia(this.userMedia?.clone(),this.screenShare?.clone(), this.dataChannelOptions);
}
dispose() {
getStreamAudioTrack(this.userMedia)?.stop();
getStreamVideoTrack(this.userMedia)?.stop();
getStreamVideoTrack(this.screenShare)?.stop();
}
}

1218
src/matrix/calls/PeerCall.ts Normal file
View File

File diff suppressed because it is too large Load Diff

225
src/matrix/calls/TODO.md Normal file
View File

@ -0,0 +1,225 @@
- relevant MSCs next to spec:
- https://github.com/matrix-org/matrix-doc/pull/2746 Improved Signalling for 1:1 VoIP
- https://github.com/matrix-org/matrix-doc/pull/2747 Transferring VoIP Calls
- https://github.com/matrix-org/matrix-doc/pull/3077 Support for multi-stream VoIP
- https://github.com/matrix-org/matrix-doc/pull/3086 Asserted identity on VoIP calls
- https://github.com/matrix-org/matrix-doc/pull/3291 Muting in VoIP calls
- https://github.com/matrix-org/matrix-doc/pull/3401 Native Group VoIP Signalling
## TODO
- DONE: implement receiving hangup
- DONE: implement cloning the localMedia so it works in safari?
- DONE: implement 3 retries per peer
- DONE: implement muting tracks with m.call.sdp_stream_metadata_changed
- DONE: implement renegotiation
- DONE: finish session id support
- call peers are essentially identified by (userid, deviceid, sessionid). If see a new session id, we first disconnect from the current member so we're ready to connect with a clean slate again (in a member event, also in to_device? no harm I suppose, given olm encryption ensures you can't spoof the deviceid).
- DONE: making logging better
- figure out why sometimes leave button does not work
- get correct members and avatars in call
- improve UI while in a call
- allow toggling audio
- support active speaker, sort speakers by last active
- close muted media stream after a while
- support highlight mode where we show active speaker and thumbnails for other participants
- better grid mode:
- we report the call view size to the view model with ResizeObserver, we calculate the A/R
- we calculate the grid based on view A/R, taking into account minimal stream size
- show name on stream view
- when you start a call, or join one, first you go to a SelectCallMedia screen where you can pick whether you want to use camera, audio or both:
- if you are joining a call, we'll default to the call intent
- if you are creating a call, we'll default to video
- when creating a call, adjust the navigation path to room/room_id/call
- when selecting a call, adjust the navigation path to room/room_id/call/call_id
- implement to_device messages arriving before m.call(.member) state event
- DONE for m.call.member, not for m.call and not for to_device other than m.call.invite arriving before invite
- reeable crypto & implement fetching olm keys before sending encrypted signalling message
- local echo for join/leave buttons?
- batch outgoing to_device messages in one request to homeserver for operations that will send out an event to all participants (e.g. mute)
- implement call ringing and rejecting a ringing call
- support screen sharing
- add button to enable, disable
- support showing stream view with large screen share video element and small camera video element (if present)
- don't load all members when loading calls to know whether they are ringing and joined by ourself
- only load our own member once, then have a way to load additional members on a call.
- see if we remove partyId entirely, it is only used for detecting remote echo which is not an issue for group calls? see https://github.com/matrix-org/matrix-spec-proposals/blob/dbkr/msc2746/proposals/2746-reliable-voip.md#add-party_id-to-all-voip-events
- remove PeerCall.waitForState ?
- invite glare is completely untested, does it work?
- how to remove call from m.call.member when just closing client?
- when closing client and still in call, tell service worker to send event on our behalf?
```js
// dispose when leaving call
this.track(platform.registerExitHandler(unloadActions => {
// batch requests will resolve immediately,
// so we can reuse the same send code that does awaits without awaiting?
const batch = new RequestBatch();
const hsApi = this.hsApi.withBatch(batch);
// _leaveCallMemberContent will need to become sync,
// so we'll need to keep track of own member event rather than rely on storage
hsApi.sendStateEvent("m.call.member", this._leaveCallMemberContent());
// does this internally: serviceWorkerHandler.trySend("sendRequestBatch", batch.toJSON());
unloadActions.sendRequestBatch(batch);
}));
```
## TODO (old)
- DONE: PeerCall
- send invite
- implement terminate
- implement waitForState
- find out if we need to do something different when renegotation is triggered (a subsequent onnegotiationneeded event) whether
we sent the invite/offer or answer. e.g. do we always do createOffer/setLocalDescription and then send it over a matrix negotiation event? even if we before called createAnswer.
- handle receiving offer and send anwser
- handle sending ice candidates
- handle ice candidates finished (iceGatheringState === 'complete')
- handle receiving ice candidates
- handle sending renegotiation
- handle receiving renegotiation
- reject call
- hangup call
- handle muting tracks
- handle remote track being muted
- handle adding/removing tracks to an ongoing call
- handle sdp metadata
- DONE: Participant
- handle glare
- encrypt to_device message with olm
- batch outgoing to_device messages in one request to homeserver for operations that will send out an event to all participants (e.g. mute)
- find out if we should start muted or not?
## Store ongoing calls
DONE: Add store with all ongoing calls so when we quit and start again, we don't have to go through all the past calls to know which ones might still be ongoing.
## Notes
we send m.call as state event in room
we add m.call.participant for our own device
we wait for other participants to add their user and device (in the sources)
for each (userid, deviceid)
- if userId < ourUserId
- get local media
- we setup a peer connection
- add local tracks
- we wait for negotation event to get sdp
- peerConn.createOffer
- peerConn.setLocalDescription
- we send an m.call.invite
- else
- wait for invite from other side
on local ice candidate:
- if we haven't ... sent invite yet? or received answer? buffer candidate
- otherwise send candidate (without buffering?)
on incoming call:
- ring, offer to answer
answering incoming call
- get local media
- peerConn.setRemoteDescription
- add local tracks to peerConn
- peerConn.createAnswer()
- peerConn.setLocalDescription
in some cases, we will actually send the invite to all devices (e.g. SFU), so
we probably still need to handle multiple anwsers?
so we would send an invite to multiple devices and pick the one for which we
received the anwser first. between invite and anwser, we could already receive
ice candidates that we need to buffer.
updating the metadata:
if we're renegotiating: use m.call.negotatie
if just muting: use m.call.sdp_stream_metadata_changed
party identification
- for 1:1 calls, we identify with a party_id
- for group calls, we identify with a device_id
## TODO
Build basic version of PeerCall
- add candidates code
DONE: Build basic version of GroupCall
- DONE: add state, block invalid actions
DONE: Make it possible to olm encrypt the messages
Do work needed for state events
- DONEish: receiving (almost done?)
- DONEish: sending
logging
DONE: Expose call objects
expose volume events from audiotrack to group call
DONE: Write view model
DONE: write view
- handle glare edge-cases (not yet sent): https://spec.matrix.org/latest/client-server-api/#glare
## Calls questions
- how do we handle glare between group calls (e.g. different state events with different call ids?)
- Split up DOM part into platform code? What abstractions to choose?
Does it make sense to come up with our own API very similar to DOM api?
- what code do we copy over vs what do we implement ourselves?
- MatrixCall: perhaps we can copy it over and modify it to our needs? Seems to have a lot of edge cases implemented.
- what is partyId about?
- CallFeed: I need better understand where it is used. It's basically a wrapper around a MediaStream with volume detection. Could it make sense to put this in platform for example?
- which parts of MSC2746 are still relevant for group calls?
- which parts of MSC2747 are still relevant for group calls? it seems mostly orthogonal?
- SOLVED: how does switching channels work? This was only enabled by MSC 2746
- you do getUserMedia()/getDisplayMedia() to get the stream(s)
- you call removeTrack/addTrack on the peerConnection
- you receive a negotiationneeded event
- you call createOffer
- you send m.call.negotiate
- SOLVED: wrt to MSC2746, is the screen share track and the audio track (and video track) part of the same stream? or do screen share tracks need to go in a different stream? it sounds incompatible with the MSC2746 requirement.
- SOLVED: how does muting work? MediaStreamTrack.enabled
- SOLVED: so, what's the difference between the call_id and the conf_id in group call events?
- call_id is the specific 1:1 call, conf_id is the thing in the m.call state event key
- so a group call has a conf_id with MxN peer calls, each having their call_id.
I think we need to synchronize the negotiation needed because we don't use a CallState to guard it...
## Thursday 3-3 notes
we probably best keep the perfect negotiation flags, as they are needed for both starting the call AND renegotiation? if only for the former, it would make sense as it is a step in setting up the call, but if the call is ongoing, does it make sense to have a MakingOffer state? it actually looks like they are only needed for renegotiation! for call setup we compare the call_ids. What does that mean for these flags?
## Peer call state transitions
FROM CALLER FROM CALLEE
Fledgling Fledgling
V `call()` V `handleInvite()`: setRemoteDescription(event.offer), add buffered candidates
V Ringing
V V `answer()`
CreateOffer V
V add local tracks V
V wait for negotionneeded events V add local tracks
V setLocalDescription() CreateAnswer
V send invite event V setLocalDescription(createAnswer())
InviteSent |
V receive anwser, setRemoteDescription() |
\___________________________________________________/
V
Connecting
V receive ice candidates and iceConnectionState becomes 'connected'
Connected
V `hangup()` or some terminate condition
Ended
so if we don't want to bother with having two call objects, we can make the existing call hangup his old call_id? That way we keep the old peerConnection.
when glare, won't we drop both calls? No: https://github.com/matrix-org/matrix-spec-proposals/pull/2746#discussion_r819388754

222
src/matrix/calls/TurnServerSource.ts Normal file
View File

@ -0,0 +1,222 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {RetainedObservableValue} from "../../observable/value";
import type {HomeServerApi} from "../net/HomeServerApi";
import type {IHomeServerRequest} from "../net/HomeServerRequest";
import type {BaseObservableValue, ObservableValue} from "../../observable/value";
import type {Clock, Timeout} from "../../platform/web/dom/Clock";
import type {ILogItem} from "../../logging/types";
type TurnServerSettings = {
uris: string[],
username: string,
password: string,
ttl: number
};
const DEFAULT_TTL = 5 * 60; // 5min
const DEFAULT_SETTINGS: RTCIceServer = {
urls: ["stun:turn.matrix.org"],
username: "",
credential: "",
};
export class TurnServerSource {
private currentObservable?: ObservableValue<RTCIceServer>;
private pollTimeout?: Timeout;
private pollRequest?: IHomeServerRequest;
private isPolling = false;
constructor(
private hsApi: HomeServerApi,
private clock: Clock,
private defaultSettings: RTCIceServer = DEFAULT_SETTINGS
) {}
getSettings(log: ILogItem): Promise<BaseObservableValue<RTCIceServer>> {
return log.wrap("get turn server", async log => {
if (!this.isPolling) {
const settings = await this.doRequest(log);
const iceServer = settings ? toIceServer(settings) : this.defaultSettings;
log.set("iceServer", iceServer);
if (this.currentObservable) {
this.currentObservable.set(iceServer);
} else {
this.currentObservable = new RetainedObservableValue(iceServer,
() => {
this.stopPollLoop();
},
() => {
// start loop on first subscribe
this.runLoop(settings?.ttl ?? DEFAULT_TTL);
});
}
}
return this.currentObservable!;
});
}
private async runLoop(initialTtl: number): Promise<void> {
let ttl = initialTtl;
this.isPolling = true;
while(this.isPolling) {
try {
this.pollTimeout = this.clock.createTimeout(ttl * 1000);
await this.pollTimeout.elapsed();
this.pollTimeout = undefined;
const settings = await this.doRequest(undefined);
if (settings) {
const iceServer = toIceServer(settings);
if (shouldUpdate(this.currentObservable!, iceServer)) {
this.currentObservable!.set(iceServer);
}
if (settings.ttl > 0) {
ttl = settings.ttl;
} else {
// stop polling is settings are good indefinitely
this.stopPollLoop();
}
} else {
ttl = DEFAULT_TTL;
}
} catch (err) {
if (err.name === "AbortError") {
/* ignore, the loop will exit because isPolling is false */
} else {
// TODO: log error
}
}
}
}
private async doRequest(log: ILogItem | undefined): Promise<TurnServerSettings | undefined> {
try {
this.pollRequest = this.hsApi.getTurnServer({log});
const settings = await this.pollRequest.response();
return settings;
} catch (err) {
if (err.name === "HomeServerError") {
return undefined;
}
throw err;
} finally {
this.pollRequest = undefined;
}
}
private stopPollLoop() {
this.isPolling = false;
this.currentObservable = undefined;
this.pollTimeout?.dispose();
this.pollTimeout = undefined;
this.pollRequest?.abort();
this.pollRequest = undefined;
}
dispose() {
this.stopPollLoop();
}
}
function shouldUpdate(observable: BaseObservableValue<RTCIceServer | undefined>, settings: RTCIceServer): boolean {
const currentSettings = observable.get();
if (!currentSettings) {
return true;
}
// same length and new settings doesn't contain any uri the old settings don't contain
const currentUrls = Array.isArray(currentSettings.urls) ? currentSettings.urls : [currentSettings.urls];
const newUrls = Array.isArray(settings.urls) ? settings.urls : [settings.urls];
const arraysEqual = currentUrls.length === newUrls.length &&
!newUrls.some(uri => !currentUrls.includes(uri));
return !arraysEqual || settings.username !== currentSettings.username ||
settings.credential !== currentSettings.credential;
}
function toIceServer(settings: TurnServerSettings): RTCIceServer {
return {
urls: settings.uris,
username: settings.username,
credential: settings.password,
credentialType: "password"
}
}
export function tests() {
return {
"shouldUpdate returns false for same object": assert => {
const observable = {get() {
return {
urls: ["a", "b"],
username: "alice",
credential: "f00",
};
}};
const same = {
urls: ["a", "b"],
username: "alice",
credential: "f00",
};
assert.equal(false, shouldUpdate(observable as any as BaseObservableValue<RTCIceServer>, same));
},
"shouldUpdate returns true for 1 different uri": assert => {
const observable = {get() {
return {
urls: ["a", "c"],
username: "alice",
credential: "f00",
};
}};
const same = {
urls: ["a", "b"],
username: "alice",
credential: "f00",
};
assert.equal(true, shouldUpdate(observable as any as BaseObservableValue<RTCIceServer>, same));
},
"shouldUpdate returns true for different user": assert => {
const observable = {get() {
return {
urls: ["a", "b"],
username: "alice",
credential: "f00",
};
}};
const same = {
urls: ["a", "b"],
username: "bob",
credential: "f00",
};
assert.equal(true, shouldUpdate(observable as any as BaseObservableValue<RTCIceServer>, same));
},
"shouldUpdate returns true for different password": assert => {
const observable = {get() {
return {
urls: ["a", "b"],
username: "alice",
credential: "f00",
};
}};
const same = {
urls: ["a", "b"],
username: "alice",
credential: "b4r",
};
assert.equal(true, shouldUpdate(observable as any as BaseObservableValue<RTCIceServer>, same));
}
}
}

240
src/matrix/calls/callEventTypes.ts Normal file
View File

@ -0,0 +1,240 @@
// allow non-camelcase as these are events type that go onto the wire
/* eslint-disable camelcase */
import type {StateEvent} from "../storage/types";
import type {SessionDescription} from "../../platform/types/WebRTC";
export enum EventType {
GroupCall = "org.matrix.msc3401.call",
GroupCallMember = "org.matrix.msc3401.call.member",
Invite = "m.call.invite",
Candidates = "m.call.candidates",
Answer = "m.call.answer",
Hangup = "m.call.hangup",
Reject = "m.call.reject",
SelectAnswer = "m.call.select_answer",
Negotiate = "m.call.negotiate",
SDPStreamMetadataChanged = "m.call.sdp_stream_metadata_changed",
SDPStreamMetadataChangedPrefix = "org.matrix.call.sdp_stream_metadata_changed",
Replaces = "m.call.replaces",
AssertedIdentity = "m.call.asserted_identity",
AssertedIdentityPrefix = "org.matrix.call.asserted_identity",
}
// TODO: Change to "sdp_stream_metadata" when MSC3077 is merged
export const SDPStreamMetadataKey = "org.matrix.msc3077.sdp_stream_metadata";
export interface FocusConfig {
user_id: string,
device_id: string
}
export interface CallDeviceMembership {
device_id: string,
session_id: string,
["expires_ts"]?: number,
feeds?: Array<{purpose: string}>
["m.foci.active"]?: Array<FocusConfig>
}
export interface CallMembership {
["m.call_id"]: string,
["m.devices"]: CallDeviceMembership[]
}
export interface CallMemberContent {
["m.calls"]: CallMembership[];
}
export enum SDPStreamMetadataPurpose {
Usermedia = "m.usermedia",
Screenshare = "m.screenshare",
}
export interface SDPStreamMetadataObject {
purpose: SDPStreamMetadataPurpose;
audio_muted: boolean;
video_muted: boolean;
}
export interface SDPStreamMetadata {
[key: string]: SDPStreamMetadataObject;
}
export interface CallCapabilities {
'm.call.transferee': boolean;
'm.call.dtmf': boolean;
}
export interface CallReplacesTarget {
id: string;
display_name: string;
avatar_url: string;
}
export type MCallBase = {
call_id: string;
version: string | number;
}
export type MGroupCallBase = MCallBase & {
conf_id: string;
device_id: string;
sender_session_id: string;
dest_session_id: string;
party_id: string; // Should not need this?
seq: number;
}
export type MCallAnswer<Base extends MCallBase> = Base & {
answer: SessionDescription;
capabilities?: CallCapabilities;
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallSelectAnswer<Base extends MCallBase> = Base & {
selected_party_id: string;
}
export type MCallInvite<Base extends MCallBase> = Base & {
offer: SessionDescription;
lifetime: number;
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallNegotiate<Base extends MCallBase> = Base & {
description: SessionDescription;
lifetime: number;
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallSDPStreamMetadataChanged<Base extends MCallBase> = Base & {
[SDPStreamMetadataKey]: SDPStreamMetadata;
}
export type MCallReplacesEvent<Base extends MCallBase> = Base & {
replacement_id: string;
target_user: CallReplacesTarget;
create_call: string;
await_call: string;
target_room: string;
}
export type MCAllAssertedIdentity<Base extends MCallBase> = Base & {
asserted_identity: {
id: string;
display_name: string;
avatar_url: string;
};
}
export type MCallCandidates<Base extends MCallBase> = Base & {
candidates: RTCIceCandidate[];
}
export type MCallHangupReject<Base extends MCallBase> = Base & {
reason?: CallErrorCode;
}
export enum CallErrorCode {
/** The user chose to end the call */
UserHangup = 'user_hangup',
/** An error code when the local client failed to create an offer. */
LocalOfferFailed = 'local_offer_failed',
/**
* An error code when there is no local mic/camera to use. This may be because
* the hardware isn't plugged in, or the user has explicitly denied access.
*/
NoUserMedia = 'no_user_media',
/**
* Error code used when a call event failed to send
* because unknown devices were present in the room
*/
UnknownDevices = 'unknown_devices',
/**
* Error code used when we fail to send the invite
* for some reason other than there being unknown devices
*/
SendInvite = 'send_invite',
/**
* An answer could not be created
*/
CreateAnswer = 'create_answer',
/**
* Error code used when we fail to send the answer
* for some reason other than there being unknown devices
*/
SendAnswer = 'send_answer',
/**
* The session description from the other side could not be set
*/
SetRemoteDescription = 'set_remote_description',
/**
* The session description from this side could not be set
*/
SetLocalDescription = 'set_local_description',
/**
* A different device answered the call
*/
AnsweredElsewhere = 'answered_elsewhere',
/**
* No media connection could be established to the other party
*/
IceFailed = 'ice_failed',
/**
* The invite timed out whilst waiting for an answer
*/
InviteTimeout = 'invite_timeout',
/**
* The call was replaced by another call
*/
Replaced = 'replaced',
/**
* Signalling for the call could not be sent (other than the initial invite)
*/
SignallingFailed = 'signalling_timeout',
/**
* The remote party is busy
*/
UserBusy = 'user_busy',
/**
* We transferred the call off to somewhere else
*/
Transfered = 'transferred',
/**
* A call from the same user was found with a new session id
*/
NewSession = 'new_session',
}
export type SignallingMessage<Base extends MCallBase> =
{type: EventType.Invite, content: MCallInvite<Base>} |
{type: EventType.Negotiate, content: MCallNegotiate<Base>} |
{type: EventType.Answer, content: MCallAnswer<Base>} |
{type: EventType.SDPStreamMetadataChanged | EventType.SDPStreamMetadataChangedPrefix, content: MCallSDPStreamMetadataChanged<Base>} |
{type: EventType.Candidates, content: MCallCandidates<Base>} |
{type: EventType.Hangup | EventType.Reject, content: MCallHangupReject<Base>};
export enum CallIntent {
Ring = "m.ring",
Prompt = "m.prompt",
Room = "m.room",
};
export enum CallType {
Video = "m.video",
Voice = "m.voice",
}

87
src/matrix/calls/common.ts Normal file
View File

@ -0,0 +1,87 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ILogItem} from "../../logging/types";
import type {Track, Stream} from "../../platform/types/MediaDevices";
import {LocalMedia} from "./LocalMedia";
export function getStreamAudioTrack(stream: Stream | undefined): Track | undefined {
return stream?.getAudioTracks()[0];
}
export function getStreamVideoTrack(stream: Stream | undefined): Track | undefined {
return stream?.getVideoTracks()[0];
}
export function mute(localMedia: LocalMedia, localMuteSettings: MuteSettings, log: ILogItem) {
return log.wrap("mute", log => {
log.set("cameraMuted", localMuteSettings.camera);
log.set("microphoneMuted", localMuteSettings.microphone);
// Mute audio
const userMediaAudio = getStreamAudioTrack(localMedia.userMedia);
if (userMediaAudio) {
const enabled = !localMuteSettings.microphone;
log.set("microphone enabled", enabled);
userMediaAudio.enabled = enabled;
}
// Mute video
const userMediaVideo = getStreamVideoTrack(localMedia.userMedia);
if (userMediaVideo) {
const enabled = !localMuteSettings.camera;
log.set("camera enabled", enabled);
userMediaVideo.enabled = enabled;
}
});
}
export class MuteSettings {
constructor (
private readonly isMicrophoneMuted: boolean = false,
private readonly isCameraMuted: boolean = false,
private hasMicrophoneTrack: boolean = false,
private hasCameraTrack: boolean = false,
) {}
updateTrackInfo(userMedia: Stream | undefined) {
this.hasMicrophoneTrack = !!getStreamAudioTrack(userMedia);
this.hasCameraTrack = !!getStreamVideoTrack(userMedia);
}
get microphone(): boolean {
return !this.hasMicrophoneTrack || this.isMicrophoneMuted;
}
get camera(): boolean {
return !this.hasCameraTrack || this.isCameraMuted;
}
toggleCamera(): MuteSettings {
return new MuteSettings(this.microphone, !this.camera, this.hasMicrophoneTrack, this.hasCameraTrack);
}
toggleMicrophone(): MuteSettings {
return new MuteSettings(!this.microphone, this.camera, this.hasMicrophoneTrack, this.hasCameraTrack);
}
equals(other: MuteSettings) {
return this.microphone === other.microphone && this.camera === other.camera;
}
}
export const CALL_LOG_TYPE = "call";
export const CALL_MEMBER_VALIDITY_PERIOD_MS = 3600 * 1000; // 1h

702
src/matrix/calls/group/GroupCall.ts Normal file
View File

@ -0,0 +1,702 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableMap} from "../../../observable/map";
import {Member, isMemberExpired, memberExpiresAt} from "./Member";
import {LocalMedia} from "../LocalMedia";
import {MuteSettings, CALL_LOG_TYPE, CALL_MEMBER_VALIDITY_PERIOD_MS, mute} from "../common";
import {MemberChange, RoomMember} from "../../room/members/RoomMember";
import {EventEmitter} from "../../../utils/EventEmitter";
import {EventType, CallIntent, CallType} from "../callEventTypes";
import { ErrorBoundary } from "../../../utils/ErrorBoundary";
import type {Options as MemberOptions} from "./Member";
import type {TurnServerSource} from "../TurnServerSource";
import type {BaseObservableMap} from "../../../observable/map";
import type {Track} from "../../../platform/types/MediaDevices";
import type {SignallingMessage, MGroupCallBase, CallMembership, CallMemberContent, CallDeviceMembership} from "../callEventTypes";
import type {Room} from "../../room/Room";
import type {StateEvent} from "../../storage/types";
import type {Platform} from "../../../platform/web/Platform";
import type {EncryptedMessage} from "../../e2ee/olm/Encryption";
import type {ILogItem, ILogger} from "../../../logging/types";
import type {Storage} from "../../storage/idb/Storage";
import type {BaseObservableValue} from "../../../observable/value";
import type {Clock, Timeout} from "../../../platform/web/dom/Clock";
export enum GroupCallState {
Fledgling = "fledgling",
Creating = "creating",
Created = "created",
Joining = "joining",
Joined = "joined",
}
function getMemberKey(userId: string, deviceId: string) {
return JSON.stringify(userId)+`,`+JSON.stringify(deviceId);
}
function memberKeyIsForUser(key: string, userId: string) {
return key.startsWith(JSON.stringify(userId)+`,`);
}
function getDeviceFromMemberKey(key: string): string {
return JSON.parse(`[${key}]`)[1];
}
export type Options = Omit<MemberOptions, "emitUpdate" | "confId" | "encryptDeviceMessage" | "turnServer"> & {
emitUpdate: (call: GroupCall, params?: any) => void;
encryptDeviceMessage: (roomId: string, userId: string, deviceId: string, message: SignallingMessage<MGroupCallBase>, log: ILogItem) => Promise<EncryptedMessage | undefined>,
storage: Storage,
random: () => number,
logger: ILogger,
turnServerSource: TurnServerSource
};
class JoinedData {
public renewMembershipTimeout?: Timeout;
constructor(
public readonly logItem: ILogItem,
public readonly membersLogItem: ILogItem,
public localMedia: LocalMedia,
public localPreviewMedia: LocalMedia,
public localMuteSettings: MuteSettings,
public readonly turnServer: BaseObservableValue<RTCIceServer>
) {}
dispose() {
this.localMedia.dispose();
this.localPreviewMedia.dispose();
this.logItem.finish();
this.renewMembershipTimeout?.dispose();
}
}
export class GroupCall extends EventEmitter<{change: never}> {
private readonly _members: ObservableMap<string, Member> = new ObservableMap();
private _memberOptions: MemberOptions;
private _state: GroupCallState;
private bufferedDeviceMessages = new Map<string, Set<SignallingMessage<MGroupCallBase>>>();
/** Set between calling join and leave. */
private joinedData?: JoinedData;
private errorBoundary = new ErrorBoundary(err => {
this.emitChange();
if (this.joinedData) {
// in case the error happens in code that does not log,
// log it here to make sure it isn't swallowed
this.joinedData.logItem.log("error at boundary").catch(err);
console.error(err);
}
});
constructor(
public readonly id: string,
public readonly isLoadedFromStorage: boolean,
newCall: boolean,
private startTime: number | undefined,
private callContent: Record<string, any>,
public readonly roomId: string,
private readonly options: Options,
) {
super();
this._state = newCall ? GroupCallState.Fledgling : GroupCallState.Created;
this._memberOptions = Object.assign({}, options, {
confId: this.id,
emitUpdate: member => {
const memberKey = getMemberKey(member.userId, member.deviceId);
// only remove expired members to whom we're not already connected
if (member.isExpired && !member.isConnected) {
const logItem = this.options.logger.log({
l: "removing expired member from call",
memberKey,
callId: this.id
})
member.logItem?.refDetached(logItem);
member.dispose();
this._members.remove(memberKey);
} else {
this._members.update(memberKey);
}
},
encryptDeviceMessage: (userId: string, deviceId: string, message: SignallingMessage<MGroupCallBase>, log) => {
return this.options.encryptDeviceMessage(this.roomId, userId, deviceId, message, log);
}
});
}
get localMedia(): LocalMedia | undefined { return this.joinedData?.localMedia; }
get localPreviewMedia(): LocalMedia | undefined { return this.joinedData?.localPreviewMedia; }
get members(): BaseObservableMap<string, Member> { return this._members; }
get isTerminated(): boolean {
return !!this.callContent?.["m.terminated"];
}
get usesFoci(): boolean {
for (const member of this._members.values()) {
if (member.usesFoci) {
return true;
}
}
return false;
}
get duration(): number | undefined {
if (typeof this.startTime === "number") {
return (this.options.clock.now() - this.startTime);
}
}
get isRinging(): boolean {
return this._state === GroupCallState.Created && this.intent === "m.ring" && !this.isMember(this.options.ownUserId);
}
get name(): string {
return this.callContent?.["m.name"];
}
get intent(): CallIntent {
return this.callContent?.["m.intent"];
}
get type(): CallType {
return this.callContent?.["m.type"];
}
/**
* Gives access the log item for this call while joined.
* Can be used for call diagnostics while in the call.
**/
get logItem(): ILogItem | undefined {
return this.joinedData?.logItem;
}
get error(): Error | undefined {
return this.errorBoundary.error;
}
join(localMedia: LocalMedia, log?: ILogItem): Promise<void> {
return this.options.logger.wrapOrRun(log, "Call.join", async joinLog => {
if (this._state !== GroupCallState.Created || this.joinedData || this.usesFoci) {
localMedia.dispose();
return;
}
const logItem = this.options.logger.child({
l: "Call.connection",
t: CALL_LOG_TYPE,
id: this.id,
ownSessionId: this.options.sessionId
});
const turnServer = await this.options.turnServerSource.getSettings(logItem);
const membersLogItem = logItem.child("member connections");
const localMuteSettings = new MuteSettings();
localMuteSettings.updateTrackInfo(localMedia.userMedia);
const localPreviewMedia = localMedia.asPreview();
const joinedData = new JoinedData(
logItem,
membersLogItem,
localMedia,
localPreviewMedia,
localMuteSettings,
turnServer
);
this.joinedData = joinedData;
await joinedData.logItem.wrap("join", async log => {
joinLog.refDetached(log);
this._state = GroupCallState.Joining;
this.emitChange();
await log.wrap("update member state", async log => {
const memberContent = await this._createMemberPayload(true);
log.set("payload", memberContent);
// send m.call.member state event
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCallMember, this.options.ownUserId, memberContent, {log});
await request.response();
this.emitChange();
});
// send invite to all members that are < my userId
for (const member of this._members.values()) {
this.connectToMember(member, joinedData, log);
}
});
});
}
async setMedia(localMedia: LocalMedia): Promise<void> {
if ((this._state === GroupCallState.Joining || this._state === GroupCallState.Joined) && this.joinedData) {
const oldMedia = this.joinedData.localMedia;
this.joinedData.localMedia = localMedia;
this.joinedData.localPreviewMedia?.dispose();
this.joinedData.localPreviewMedia = localMedia.asPreview();
// reflect the fact we gained or lost local tracks in the local mute settings
// and update the track info so PeerCall can use it to send up to date metadata,
this.joinedData.localMuteSettings.updateTrackInfo(localMedia.userMedia);
this.emitChange(); //allow listeners to see new media/mute settings
// TODO: if setMedia fails on one of the members, we should revert to the old media
// on the members processed so far, and show an error that we could not set the new media
// for this, we will need to remove the usage of the errorBoundary in member.setMedia.
await Promise.all(Array.from(this._members.values()).map(m => {
return m.setMedia(localMedia, oldMedia);
}));
oldMedia?.dispose();
}
}
async setMuted(muteSettings: MuteSettings): Promise<void> {
const {joinedData} = this;
if (!joinedData) {
return;
}
const prevMuteSettings = joinedData.localMuteSettings;
// we still update the mute settings if nothing changed because
// you might be muted because you don't have a track or because
// you actively chosen to mute
// (which we want to respect in the future when you add a track)
muteSettings.updateTrackInfo(joinedData.localMedia.userMedia);
joinedData.localMuteSettings = muteSettings;
if (!prevMuteSettings.equals(muteSettings)) {
// Mute our copies of LocalMedias;
// otherwise the camera lights will still be on.
if (this.localPreviewMedia) {
mute(this.localPreviewMedia, muteSettings, this.joinedData!.logItem);
}
if (this.localMedia) {
mute(this.localMedia, muteSettings, this.joinedData!.logItem);
}
// TODO: if setMuted fails on one of the members, we should revert to the old media
// on the members processed so far, and show an error that we could not set the new media
// for this, we will need to remove the usage of the errorBoundary in member.setMuted.
await Promise.all(Array.from(this._members.values()).map(m => {
return m.setMuted(joinedData.localMuteSettings);
}));
this.emitChange();
}
}
get muteSettings(): MuteSettings | undefined {
return this.joinedData?.localMuteSettings;
}
get hasJoined() {
return this._state === GroupCallState.Joining || this._state === GroupCallState.Joined;
}
async leave(log?: ILogItem): Promise<void> {
await this.options.logger.wrapOrRun(log, "Call.leave", async log => {
const {joinedData} = this;
if (!joinedData) {
return;
}
try {
joinedData.renewMembershipTimeout?.dispose();
joinedData.renewMembershipTimeout = undefined;
const memberContent = await this._createMemberPayload(false);
// send m.call.member state event
if (memberContent) {
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCallMember, this.options.ownUserId, memberContent, {log});
await request.response();
// our own user isn't included in members, so not in the count
if ((this.intent === CallIntent.Ring || this.intent === CallIntent.Prompt) && this._members.size === 0) {
await this.terminate(log);
}
} else {
log.set("already_left", true);
}
} finally {
// disconnect is called both from the sync loop and from methods like this one that
// are called from the view model. We want errors during the sync loop being caught
// by the errorboundary, but since leave is called from the view model, we want
// the error to be thrown. So here we check if disconnect succeeded, and if not
// we rethrow the error put into the errorBoundary.
if(!this.disconnect(log)) {
throw this.errorBoundary.error;
}
}
});
}
private terminate(log?: ILogItem): Promise<void> {
return this.options.logger.wrapOrRun(log, {l: "terminate call", t: CALL_LOG_TYPE}, async log => {
if (this._state === GroupCallState.Fledgling) {
return;
}
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCall, this.id, Object.assign({}, this.callContent, {
"m.terminated": true
}), {log});
await request.response();
});
}
/** @internal */
create(type: CallType, log: ILogItem): Promise<void> {
return log.wrap({l: "create call", t: CALL_LOG_TYPE}, async log => {
if (this._state !== GroupCallState.Fledgling) {
return;
}
this._state = GroupCallState.Creating;
this.emitChange();
this.callContent = Object.assign({
"m.type": type,
}, this.callContent);
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCall, this.id, this.callContent!, {log});
await request.response();
this._state = GroupCallState.Created;
this.emitChange();
});
}
/** @internal */
updateCallEvent(event: StateEvent, syncLog: ILogItem) {
this.errorBoundary.try(() => {
syncLog.wrap({l: "update call", t: CALL_LOG_TYPE, id: this.id}, log => {
if (typeof this.startTime !== "number") {
this.startTime = event.origin_server_ts;
}
this.callContent = event.content;
if (this._state === GroupCallState.Creating) {
this._state = GroupCallState.Created;
}
log.set("status", this._state);
this.emitChange();
});
});
}
/** @internal */
updateRoomMembers(memberChanges: Map<string, MemberChange>) {
this.errorBoundary.try(() => {
for (const change of memberChanges.values()) {
const {member} = change;
for (const callMember of this._members.values()) {
// find all call members for a room member (can be multiple, for every device)
if (callMember.userId === member.userId) {
callMember.updateRoomMember(member);
}
}
}
});
}
/** @internal */
updateMembership(userId: string, roomMember: RoomMember, callMembership: CallMembership, syncLog: ILogItem) {
this.errorBoundary.try(async () => {
await syncLog.wrap({l: "update call membership", t: CALL_LOG_TYPE, id: this.id, userId}, async log => {
const now = this.options.clock.now();
const devices = callMembership["m.devices"];
const previousDeviceIds = this.getDeviceIdsForUserId(userId);
for (const device of devices) {
const deviceId = device.device_id;
const memberKey = getMemberKey(userId, deviceId);
if (userId === this.options.ownUserId && deviceId === this.options.ownDeviceId) {
log.wrap("update own membership", log => {
if (this.hasJoined) {
if (this.joinedData) {
this.joinedData.logItem.refDetached(log);
}
this._setupRenewMembershipTimeout(device, log);
}
if (this._state === GroupCallState.Joining) {
log.set("joined", true);
this._state = GroupCallState.Joined;
this.emitChange();
}
});
} else {
await log.wrap({l: "update device membership", id: memberKey, sessionId: device.session_id}, async log => {
if (isMemberExpired(device, now)) {
log.set("expired", true);
const member = this._members.get(memberKey);
if (member) {
member.dispose();
this._members.remove(memberKey);
log.set("removed", true);
}
return;
}
let member = this._members.get(memberKey);
const sessionIdChanged = member && member.sessionId !== device.session_id;
if (member && !sessionIdChanged) {
log.set("update", true);
member.updateCallInfo(device, log);
} else {
if (member && sessionIdChanged) {
log.set("removedSessionId", member.sessionId);
const disconnectLogItem = await member.disconnect(false);
if (disconnectLogItem) {
log.refDetached(disconnectLogItem);
}
member.dispose();
this._members.remove(memberKey);
member = undefined;
}
log.set("add", true);
member = new Member(
roomMember,
device, this._memberOptions,
log
);
this._members.add(memberKey, member);
if (this.joinedData) {
this.connectToMember(member, this.joinedData, log);
}
}
// flush pending messages, either after having created the member,
// or updated the session id with updateCallInfo
this.flushPendingIncomingDeviceMessages(member, log);
});
}
}
const newDeviceIds = new Set<string>(devices.map(call => call.device_id));
// remove user as member of any calls not present anymore
for (const previousDeviceId of previousDeviceIds) {
if (!newDeviceIds.has(previousDeviceId)) {
this.removeMemberDevice(userId, previousDeviceId, log);
}
}
if (userId === this.options.ownUserId && !newDeviceIds.has(this.options.ownDeviceId)) {
this.removeOwnDevice(log);
}
});
});
}
/** @internal */
removeMembership(userId: string, syncLog: ILogItem) {
this.errorBoundary.try(() => {
const deviceIds = this.getDeviceIdsForUserId(userId);
syncLog.wrap({
l: "remove call member",
t: CALL_LOG_TYPE,
id: this.id,
userId
}, log => {
for (const deviceId of deviceIds) {
this.removeMemberDevice(userId, deviceId, log);
}
if (userId === this.options.ownUserId) {
this.removeOwnDevice(log);
}
});
});
}
private flushPendingIncomingDeviceMessages(member: Member, log: ILogItem) {
const memberKey = getMemberKey(member.userId, member.deviceId);
const bufferedMessages = this.bufferedDeviceMessages.get(memberKey);
// check if we have any pending message for the member with (userid, deviceid, sessionid)
if (bufferedMessages) {
for (const message of bufferedMessages) {
if (message.content.sender_session_id === member.sessionId) {
member.handleDeviceMessage(message, log);
bufferedMessages.delete(message);
}
}
if (bufferedMessages.size === 0) {
this.bufferedDeviceMessages.delete(memberKey);
}
}
}
private getDeviceIdsForUserId(userId: string): string[] {
return Array.from(this._members.keys())
.filter(key => memberKeyIsForUser(key, userId))
.map(key => getDeviceFromMemberKey(key));
}
private isMember(userId: string): boolean {
return Array.from(this._members.keys()).some(key => memberKeyIsForUser(key, userId));
}
private removeOwnDevice(log: ILogItem) {
log.wrap("remove own membership", log => {
this.disconnect(log);
});
}
/** @internal */
disconnect(log: ILogItem): Promise<void> | true {
return this.errorBoundary.try(async () => {
if (this.hasJoined) {
for (const member of this._members.values()) {
const disconnectLogItem = await member.disconnect(true);
if (disconnectLogItem) {
log.refDetached(disconnectLogItem);
}
}
this._state = GroupCallState.Created;
}
this.joinedData?.dispose();
this.joinedData = undefined;
this.emitChange();
}, false) || true;
}
/** @internal */
private async removeMemberDevice(userId: string, deviceId: string, log: ILogItem) {
const memberKey = getMemberKey(userId, deviceId);
await log.wrap({l: "remove device member", id: memberKey}, async log => {
const member = this._members.get(memberKey);
if (member) {
log.set("leave", true);
const disconnectLogItem = await member.disconnect(false);
if (disconnectLogItem) {
log.refDetached(disconnectLogItem);
}
member.dispose();
this._members.remove(memberKey);
}
});
}
/** @internal */
handleDeviceMessage(message: SignallingMessage<MGroupCallBase>, userId: string, deviceId: string, syncLog: ILogItem) {
this.errorBoundary.try(() => {
// TODO: return if we are not membering to the call
const key = getMemberKey(userId, deviceId);
let member = this._members.get(key);
if (member && message.content.sender_session_id === member.sessionId) {
member.handleDeviceMessage(message, syncLog);
} else {
const item = syncLog.log({
l: "call: buffering to_device message, member not found",
t: CALL_LOG_TYPE,
id: this.id,
userId,
deviceId,
sessionId: message.content.sender_session_id,
type: message.type
});
// we haven't received the m.call.member yet for this caller (or with this session id).
// buffer the device messages or create the member/call as it should arrive in a moment
let messages = this.bufferedDeviceMessages.get(key);
if (!messages) {
messages = new Set();
this.bufferedDeviceMessages.set(key, messages);
}
messages.add(message);
}
});
}
private async _createMemberPayload(includeOwn: boolean): Promise<CallMemberContent> {
const {storage} = this.options;
const txn = await storage.readTxn([storage.storeNames.roomState]);
const stateEvent = await txn.roomState.get(this.roomId, EventType.GroupCallMember, this.options.ownUserId);
const stateContent: CallMemberContent = stateEvent?.event?.content as CallMemberContent ?? {
["m.calls"]: []
};
let callsInfo = stateContent["m.calls"];
let callInfo = callsInfo.find(c => c["m.call_id"] === this.id);
if (!callInfo) {
callInfo = {
["m.call_id"]: this.id,
["m.devices"]: []
};
callsInfo.push(callInfo);
}
const now = this.options.clock.now();
callInfo["m.devices"] = callInfo["m.devices"].filter(d => {
// remove our own device (to add it again below)
if (d["device_id"] === this.options.ownDeviceId) {
return false;
}
// also remove any expired devices (+ the validity period added again)
if (memberExpiresAt(d) === undefined || isMemberExpired(d, now, CALL_MEMBER_VALIDITY_PERIOD_MS)) {
return false;
}
return true;
});
if (includeOwn) {
callInfo["m.devices"].push({
["device_id"]: this.options.ownDeviceId,
["session_id"]: this.options.sessionId,
["expires_ts"]: now + CALL_MEMBER_VALIDITY_PERIOD_MS,
feeds: [{purpose: "m.usermedia"}]
});
}
// filter out empty call membership
stateContent["m.calls"] = callsInfo.filter(c => c["m.devices"].length !== 0);
return stateContent;
}
private async connectToMember(member: Member, joinedData: JoinedData, log: ILogItem) {
const memberKey = getMemberKey(member.userId, member.deviceId);
const logItem = joinedData.membersLogItem.child({
l: "member",
id: memberKey,
sessionId: member.sessionId
});
await log.wrap({l: "connect", id: memberKey}, async log => {
const connectItem = await member.connect(
joinedData.localMedia,
joinedData.localMuteSettings,
joinedData.turnServer,
logItem
);
if (connectItem) {
log.refDetached(connectItem);
}
});
}
protected emitChange() {
this.emit("change");
this.options.emitUpdate(this);
}
private _setupRenewMembershipTimeout(callDeviceMembership: CallDeviceMembership, log: ILogItem) {
const {joinedData} = this;
if (!joinedData) {
return;
}
joinedData.renewMembershipTimeout?.dispose();
joinedData.renewMembershipTimeout = undefined;
const expiresAt = memberExpiresAt(callDeviceMembership);
if (typeof expiresAt !== "number") {
return;
}
const expiresFromNow = expiresAt - this.options.clock.now();
// renew 1 to 5 minutes (8.3% of 1h, but min 10s) before expiring
// do it a bit beforehand and somewhat random to not collide with
// other clients trying to renew as well
const timeToRenewBeforeExpiration = Math.max(10000, Math.ceil((0.2 +(this.options.random() * 0.8)) * (0.08333 * CALL_MEMBER_VALIDITY_PERIOD_MS)));
const renewFromNow = Math.max(0, expiresFromNow - timeToRenewBeforeExpiration);
log.set("expiresIn", expiresFromNow);
log.set("renewIn", renewFromNow);
joinedData.renewMembershipTimeout = this.options.clock.createTimeout(renewFromNow);
joinedData.renewMembershipTimeout.elapsed().then(
() => {
joinedData.logItem.wrap("renew membership", async log => {
const memberContent = await this._createMemberPayload(true);
log.set("payload", memberContent);
// send m.call.member state event
const request = this.options.hsApi.sendState(this.roomId, EventType.GroupCallMember, this.options.ownUserId, memberContent, {log});
await request.response();
});
},
() => { /* assume we're swallowing AbortError from dispose above */ }
);
}
dispose() {
this.joinedData?.dispose();
for (const member of this._members.values()) {
member.dispose();
}
}
}

587
src/matrix/calls/group/Member.ts Normal file
View File

@ -0,0 +1,587 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {PeerCall, CallState, IncomingMessageAction} from "../PeerCall";
import {makeTxnId, makeId} from "../../common";
import {EventType, CallErrorCode} from "../callEventTypes";
import {formatToDeviceMessagesPayload} from "../../common";
import {sortedIndex} from "../../../utils/sortedIndex";
import { ErrorBoundary } from "../../../utils/ErrorBoundary";
import {MuteSettings} from "../common";
import type {Options as PeerCallOptions, RemoteMedia} from "../PeerCall";
import type {LocalMedia} from "../LocalMedia";
import type {HomeServerApi} from "../../net/HomeServerApi";
import type {MCallBase, MGroupCallBase, SignallingMessage, CallDeviceMembership} from "../callEventTypes";
import type {GroupCall} from "./GroupCall";
import {RoomMember} from "../../room/members/RoomMember";
import type {EncryptedMessage} from "../../e2ee/olm/Encryption";
import type {ILogItem} from "../../../logging/types";
import type {BaseObservableValue} from "../../../observable/value";
import type {Clock, Timeout} from "../../../platform/web/dom/Clock";
export type Options = Omit<PeerCallOptions, "emitUpdate" | "sendSignallingMessage" | "turnServer"> & {
confId: string,
ownUserId: string,
ownDeviceId: string,
// local session id of our client
sessionId: string,
hsApi: HomeServerApi,
encryptDeviceMessage: (userId: string, deviceId: string, message: SignallingMessage<MGroupCallBase>, log: ILogItem) => Promise<EncryptedMessage | undefined>,
emitUpdate: (participant: Member, params?: any) => void,
clock: Clock
}
const errorCodesWithoutRetry = [
CallErrorCode.UserHangup,
CallErrorCode.AnsweredElsewhere,
CallErrorCode.Replaced,
CallErrorCode.UserBusy,
CallErrorCode.Transfered,
CallErrorCode.NewSession
];
/** @internal */
class MemberConnection {
public retryCount: number = 0;
public peerCall?: PeerCall;
public lastProcessedSeqNr: number | undefined;
// XXX: Not needed anymore when seq is scoped to call_id
// see https://github.com/matrix-org/matrix-spec-proposals/pull/3401#discussion_r1097482166
public lastIgnoredSeqNr: number | undefined;
public queuedSignallingMessages: SignallingMessage<MGroupCallBase>[] = [];
public outboundSeqCounter: number = 0;
constructor(
public localMedia: LocalMedia,
public localMuteSettings: MuteSettings,
public turnServer: BaseObservableValue<RTCIceServer>,
public readonly logItem: ILogItem
) {}
get canDequeueNextSignallingMessage() {
if (this.queuedSignallingMessages.length === 0) {
return false;
}
const first = this.queuedSignallingMessages[0];
const firstSeq = first.content.seq;
// prevent not being able to jump over seq values of ignored messages for other call ids
// as they don't increase lastProcessedSeqNr.
if (this.lastIgnoredSeqNr !== undefined && firstSeq === this.lastIgnoredSeqNr + 1) {
return true;
}
if (this.lastProcessedSeqNr === undefined) {
return firstSeq === 0;
}
// allow messages with both a seq we've just seen and
// the next one to be dequeued as it can happen
// that messages for other callIds (which could repeat seq)
// are present in the queue
// XXX: Not needed anymore when seq is scoped to call_id
// see https://github.com/matrix-org/matrix-spec-proposals/pull/3401#discussion_r1097482166
return firstSeq <= (this.lastProcessedSeqNr + 1);
}
dispose() {
this.peerCall?.dispose();
this.localMedia.dispose();
this.logItem.finish();
}
}
export class Member {
private connection?: MemberConnection;
private expireTimeout?: Timeout;
private errorBoundary = new ErrorBoundary(err => {
this.options.emitUpdate(this, "error");
if (this.connection) {
// in case the error happens in code that does not log,
// log it here to make sure it isn't swallowed
this.connection.logItem.log("error at boundary").catch(err);
}
});
constructor(
public member: RoomMember,
private callDeviceMembership: CallDeviceMembership,
private options: Options,
updateMemberLog: ILogItem
) {
this._renewExpireTimeout(updateMemberLog);
}
get error(): Error | undefined {
return this.errorBoundary.error;
}
get usesFoci(): boolean {
const activeFoci = this.callDeviceMembership["m.foci.active"];
return Array.isArray(activeFoci) && activeFoci.length > 0;
}
private _renewExpireTimeout(log: ILogItem) {
this.expireTimeout?.dispose();
this.expireTimeout = undefined;
const expiresAt = memberExpiresAt(this.callDeviceMembership);
if (typeof expiresAt !== "number") {
return;
}
const expiresFromNow = Math.max(0, expiresAt - this.options.clock.now());
log?.set("expiresIn", expiresFromNow);
// add 10ms to make sure isExpired returns true
this.expireTimeout = this.options.clock.createTimeout(expiresFromNow + 10);
this.expireTimeout.elapsed().then(
() => { this.options.emitUpdate(this, "isExpired"); },
(err) => { /* ignore abort error */ },
);
}
/**
* Gives access the log item for this item once joined to the group call.
* The signalling for this member will be log in this item.
* Can be used for call diagnostics while in the call.
**/
get logItem(): ILogItem | undefined {
return this.connection?.logItem;
}
get remoteMedia(): RemoteMedia | undefined {
return this.connection?.peerCall?.remoteMedia;
}
get isExpired(): boolean {
// never consider a peer we're connected to, to be expired
return !this.isConnected && isMemberExpired(this.callDeviceMembership, this.options.clock.now());
}
get remoteMuteSettings(): MuteSettings | undefined {
return this.connection?.peerCall?.remoteMuteSettings;
}
get isConnected(): boolean {
return this.connection?.peerCall?.state === CallState.Connected;
}
get userId(): string {
return this.member.userId;
}
get deviceId(): string {
return this.callDeviceMembership.device_id;
}
/** session id of the member */
get sessionId(): string {
return this.callDeviceMembership.session_id;
}
get dataChannel(): any | undefined {
return this.connection?.peerCall?.dataChannel;
}
/** @internal */
connect(localMedia: LocalMedia, localMuteSettings: MuteSettings, turnServer: BaseObservableValue<RTCIceServer>, memberLogItem: ILogItem): Promise<ILogItem | undefined> | undefined {
return this.errorBoundary.try(async () => {
if (this.connection) {
return;
}
// Safari can't send a MediaStream to multiple sources, so clone it
const connection = new MemberConnection(
localMedia.clone(),
localMuteSettings,
turnServer,
memberLogItem
);
this.connection = connection;
let connectLogItem: ILogItem | undefined;
await connection.logItem.wrap("connect", async log => {
connectLogItem = log;
await this.callIfNeeded(log);
});
return connectLogItem;
});
}
private callIfNeeded(log: ILogItem): Promise<void> {
return log.wrap("callIfNeeded", async log => {
// otherwise wait for it to connect
let shouldInitiateCall;
// the lexicographically lower side initiates the call
if (this.member.userId === this.options.ownUserId) {
shouldInitiateCall = this.deviceId > this.options.ownDeviceId;
} else {
shouldInitiateCall = this.member.userId > this.options.ownUserId;
}
if (shouldInitiateCall) {
const connection = this.connection!;
connection.peerCall = this._createPeerCall(makeId("c"));
await connection.peerCall.call(
connection.localMedia,
connection.localMuteSettings,
log
);
} else {
log.set("wait_for_invite", true);
}
});
}
/** @internal */
disconnect(hangup: boolean): Promise<ILogItem | undefined> | undefined {
return this.errorBoundary.try(async () => {
const {connection} = this;
if (!connection) {
return;
}
let disconnectLogItem: ILogItem | undefined;
// if if not sending the hangup, still log disconnect
await connection.logItem.wrap("disconnect", async log => {
disconnectLogItem = log;
if (hangup && connection.peerCall) {
await connection.peerCall.hangup(CallErrorCode.UserHangup, log);
}
});
connection.dispose();
this.connection = undefined;
return disconnectLogItem;
});
}
/** @internal */
updateCallInfo(callDeviceMembership: CallDeviceMembership, causeItem: ILogItem) {
this.errorBoundary.try(() => {
this.callDeviceMembership = callDeviceMembership;
this._renewExpireTimeout(causeItem);
if (this.connection) {
this.connection.logItem.refDetached(causeItem);
}
});
}
/** @internal */
updateRoomMember(roomMember: RoomMember) {
this.member = roomMember;
// TODO: this emits an update during the writeSync phase, which we usually try to avoid
this.options.emitUpdate(this);
}
/** @internal */
emitUpdateFromPeerCall = async (peerCall: PeerCall, params: any, log: ILogItem): Promise<void> => {
const connection = this.connection!;
if (peerCall.state === CallState.Ringing) {
connection.logItem.wrap("ringing, answer peercall", answerLog => {
log.refDetached(answerLog);
return peerCall.answer(connection.localMedia, connection.localMuteSettings, answerLog);
});
}
else if (peerCall.state === CallState.Ended) {
const hangupReason = peerCall.hangupReason;
peerCall.dispose();
connection.peerCall = undefined;
if (hangupReason && !errorCodesWithoutRetry.includes(hangupReason)) {
connection.retryCount += 1;
const {retryCount} = connection;
await connection.logItem.wrap({l: "retry connection", retryCount}, async retryLog => {
log.refDetached(retryLog);
if (retryCount <= 3) {
await this.callIfNeeded(retryLog);
} else {
const disconnectLogItem = await this.disconnect(false);
if (disconnectLogItem) {
retryLog.refDetached(disconnectLogItem);
}
}
});
}
}
this.options.emitUpdate(this, params);
}
/** @internal */
sendSignallingMessage = async (message: SignallingMessage<MCallBase>, log: ILogItem): Promise<void> => {
const groupMessage = message as SignallingMessage<MGroupCallBase>;
groupMessage.content.seq = this.connection!.outboundSeqCounter++;
groupMessage.content.conf_id = this.options.confId;
groupMessage.content.device_id = this.options.ownDeviceId;
groupMessage.content.party_id = this.options.ownDeviceId;
groupMessage.content.sender_session_id = this.options.sessionId;
groupMessage.content.dest_session_id = this.sessionId;
let payload;
let type: string = message.type;
const encryptedMessages = await this.options.encryptDeviceMessage(this.member.userId, this.deviceId, groupMessage, log);
if (encryptedMessages) {
payload = formatToDeviceMessagesPayload(encryptedMessages);
type = "m.room.encrypted";
} else {
// device needs deviceId and userId
payload = formatToDeviceMessagesPayload([{content: groupMessage.content, device: this}]);
}
// TODO: remove this for release
log.set("payload", groupMessage.content);
const request = this.options.hsApi.sendToDevice(
type,
payload,
makeTxnId(),
{log}
);
await request.response();
}
/** @internal */
handleDeviceMessage(message: SignallingMessage<MGroupCallBase>, syncLog: ILogItem): void {
this.errorBoundary.try(() => {
syncLog.wrap({l: "Member.handleDeviceMessage", type: message.type, seq: message.content?.seq}, log => {
const {connection} = this;
if (connection) {
const destSessionId = message.content.dest_session_id;
if (destSessionId !== this.options.sessionId) {
const logItem = connection.logItem.log({l: "ignoring to_device event with wrong session_id", destSessionId, type: message.type});
log.refDetached(logItem);
return;
}
// if there is no peerCall, we either create it with an invite and Handle is implied or we'll ignore it
if (connection.peerCall) {
const action = connection.peerCall.getMessageAction(message);
// deal with glare and replacing the call before creating new calls
if (action === IncomingMessageAction.InviteGlare) {
const {shouldReplace, log} = connection.peerCall.handleInviteGlare(message, this.deviceId, connection.logItem);
if (log) {
log.refDetached(log);
}
if (shouldReplace) {
connection.peerCall.dispose();
connection.peerCall = undefined;
}
}
}
// create call on invite
if (message.type === EventType.Invite && !connection.peerCall) {
connection.peerCall = this._createPeerCall(message.content.call_id);
}
// enqueue
const idx = sortedIndex(connection.queuedSignallingMessages, message, (a, b) => a.content.seq - b.content.seq);
connection.queuedSignallingMessages.splice(idx, 0, message);
// dequeue as much as we can
let hasNewMessageBeenDequeued = false;
if (connection.peerCall) {
hasNewMessageBeenDequeued = this.dequeueSignallingMessages(connection, connection.peerCall, message, log);
}
if (!hasNewMessageBeenDequeued) {
log.refDetached(connection.logItem.log({l: "queued message", type: message.type, seq: message.content.seq, idx}));
}
} else {
// TODO: the right thing to do here would be to at least enqueue the message rather than drop it,
// and if it's up to the other end to send the invite and the type is an invite to actually
// call connect and assume the m.call.member state update is on its way?
syncLog.log({l: "member not connected", userId: this.userId, deviceId: this.deviceId});
}
});
});
}
private dequeueSignallingMessages(connection: MemberConnection, peerCall: PeerCall, newMessage: SignallingMessage<MGroupCallBase>, syncLog: ILogItem): boolean {
let hasNewMessageBeenDequeued = false;
while (connection.canDequeueNextSignallingMessage) {
const message = connection.queuedSignallingMessages.shift()!;
const isNewMsg = message === newMessage;
hasNewMessageBeenDequeued = hasNewMessageBeenDequeued || isNewMsg;
syncLog.wrap(isNewMsg ? "process message" : "dequeue message", log => {
const seq = message.content?.seq;
log.set("seq", seq);
log.set("type", message.type);
// ignore items in the queue that should not be handled and prevent
// the lastProcessedSeqNr being corrupted with the `seq` for other call ids
// XXX: Not needed anymore when seq is scoped to call_id
// see https://github.com/matrix-org/matrix-spec-proposals/pull/3401#discussion_r1097482166
const action = peerCall.getMessageAction(message);
if (action === IncomingMessageAction.Handle) {
const item = peerCall.handleIncomingSignallingMessage(message, this.deviceId, connection.logItem);
log.refDetached(item);
connection.lastProcessedSeqNr = seq;
} else {
log.set("ignored", true);
connection.lastIgnoredSeqNr = seq;
}
});
}
return hasNewMessageBeenDequeued;
}
/** @internal */
async setMedia(localMedia: LocalMedia, previousMedia: LocalMedia): Promise<void> {
return this.errorBoundary.try(async () => {
const {connection} = this;
if (connection) {
connection.localMedia = localMedia.replaceClone(connection.localMedia, previousMedia);
await connection.peerCall?.setMedia(connection.localMedia, connection.logItem);
}
});
}
async setMuted(muteSettings: MuteSettings): Promise<void> {
return this.errorBoundary.try(async () => {
const {connection} = this;
if (connection) {
connection.localMuteSettings = muteSettings;
await connection.peerCall?.setMuted(muteSettings, connection.logItem);
}
});
}
private _createPeerCall(callId: string): PeerCall {
const connection = this.connection!;
return new PeerCall(callId, Object.assign({}, this.options, {
errorBoundary: this.errorBoundary,
emitUpdate: this.emitUpdateFromPeerCall,
sendSignallingMessage: this.sendSignallingMessage,
turnServer: connection.turnServer
}), connection.logItem);
}
dispose() {
this.connection?.dispose();
this.connection = undefined;
this.expireTimeout?.dispose();
this.expireTimeout = undefined;
// ensure the emitUpdate callback can't be called anymore
this.options.emitUpdate = () => {};
}
}
export function memberExpiresAt(callDeviceMembership: CallDeviceMembership): number | undefined {
const expiresAt = callDeviceMembership["expires_ts"];
if (Number.isSafeInteger(expiresAt)) {
return expiresAt;
}
}
export function isMemberExpired(callDeviceMembership: CallDeviceMembership, now: number, margin: number = 0) {
const expiresAt = memberExpiresAt(callDeviceMembership);
return typeof expiresAt === "number" ? ((expiresAt + margin) <= now) : true;
}
import {ObservableValue} from "../../../observable/value";
import {Clock as MockClock} from "../../../mocks/Clock";
import {Instance as NullLoggerInstance} from "../../../logging/NullLogger";
export function tests() {
class MockMedia {
clone(): MockMedia { return this; }
}
class MockPeerConn {
addEventListener() {}
removeEventListener() {}
setConfiguration() {}
setRemoteDescription() {}
getReceivers() { return [{}]; } // non-empty array
getSenders() { return []; }
addTrack() { return {}; }
removeTrack() {}
close() {}
}
return {
"test queue doesn't get blocked by enqueued, then ignored device message": assert => {
// XXX we might want to refactor the queue code a bit so it's easier to test
// without having to provide so many mocks
const clock = new MockClock();
// setup logging
const logger = NullLoggerInstance;
// const logger = new Logger({platform: {clock, random: Math.random}});
// logger.addReporter(new ConsoleReporter());
// create member
const callDeviceMembership = {
["device_id"]: "BVPIHSKXFC",
["session_id"]: "s1d5863f41ec5a5",
["expires_ts"]: 123,
feeds: [{purpose: "m.usermedia"}]
};
const roomMember = RoomMember.fromUserId("!abc", "@bruno4:matrix.org", "join");
const turnServer = new ObservableValue({}) as ObservableValue<RTCIceServer>;
// @ts-ignore
const options = {
confId: "conf",
ownUserId: "@foobaraccount2:matrix.org",
ownDeviceId: "CMLEZSARRT",
sessionId: "s1cece7088b9d35",
clock,
emitUpdate: () => {},
webRTC: {
prepareSenderForPurpose: () => {},
createPeerConnection() {
return new MockPeerConn();
}
}
} as Options;
const member = new Member(roomMember, callDeviceMembership, options, logger.child("member"));
member.connect(new MockMedia() as LocalMedia, new MuteSettings(), turnServer, logger.child("connect"));
// pretend we've already received 3 messages
// @ts-ignore
member.connection!.lastProcessedSeqNr = 2;
// send hangup with seq=3, this will enqueue the message because there is no peerCall
// as it's up to @bruno4:matrix.org to send the invite
const hangup = {
type: EventType.Hangup,
content: {
"call_id": "c0ac1b0e37afe73",
"version": 1,
"reason": "invite_timeout",
"seq": 3,
"conf_id": "conf-16a120796440a6",
"device_id": "BVPIHSKXFC",
"party_id": "BVPIHSKXFC",
"sender_session_id": "s1d5863f41ec5a5",
"dest_session_id": "s1cece7088b9d35"
}
} as SignallingMessage<MGroupCallBase>;
member.handleDeviceMessage(hangup, logger.child("handle hangup"));
// Send an invite with seq=4, this will create a new peer call with a the call id
// when dequeueing the hangup from before, it'll get ignored because it is
// for the previous call id.
const invite = {
type: EventType.Invite,
content: {
"call_id": "c1175b12d559fb1",
"offer": {
"type": "offer",
"sdp": "..."
},
"org.matrix.msc3077.sdp_stream_metadata": {
"60087b60-487e-4fa0-8229-b232c18e1332": {
"purpose": "m.usermedia",
"audio_muted": false,
"video_muted": false
}
},
"version": 1,
"lifetime": 60000,
"seq": 4,
"conf_id": "conf-16a120796440a6",
"device_id": "BVPIHSKXFC",
"party_id": "BVPIHSKXFC",
"sender_session_id": "s1d5863f41ec5a5",
"dest_session_id": "s1cece7088b9d35"
}
} as SignallingMessage<MGroupCallBase>;
member.handleDeviceMessage(invite, logger.child("handle invite"));
// @ts-ignore
assert.equal(member.connection!.queuedSignallingMessages.length, 0);
// logger.reporters[0].printOpenItems();
}
};
}

23
src/matrix/common.js
View File

@ -15,16 +15,37 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {groupBy} from "../utils/groupBy";
export function makeTxnId() {
return makeId("t");
}
export function makeId(prefix) {
const n = Math.floor(Math.random() * Number.MAX_SAFE_INTEGER);
const str = n.toString(16);
return "t" + "0".repeat(14 - str.length) + str;
return prefix + "0".repeat(14 - str.length) + str;
}
export function isTxnId(txnId) {
return txnId.startsWith("t") && txnId.length === 15;
}
export function formatToDeviceMessagesPayload(messages) {
const messagesByUser = groupBy(messages, message => message.device.userId);
const payload = {
messages: Array.from(messagesByUser.entries()).reduce((userMap, [userId, messages]) => {
userMap[userId] = messages.reduce((deviceMap, message) => {
deviceMap[message.device.deviceId] = message.content;
return deviceMap;
}, {});
return userMap;
}, {})
};
return payload;
}
export function tests() {
return {
"isTxnId succeeds on result of makeTxnId": assert => {

8
src/matrix/e2ee/DecryptionResult.ts
View File

@ -64,6 +64,14 @@ export class DecryptionResult {
}
}
get userId(): string | undefined {
return this.device?.userId;
}
get deviceId(): string | undefined {
return this.device?.deviceId;
}
get isVerificationUnknown(): boolean {
return !this.device;
}

54
src/matrix/e2ee/DeviceTracker.js
View File

@ -399,6 +399,60 @@ export class DeviceTracker {
return this._devicesForUserIdentities(upToDateIdentities, outdatedUserIds, hsApi, log);
}
/** gets a single device */
async deviceForId(userId, deviceId, hsApi, log) {
const txn = await this._storage.readTxn([
this._storage.storeNames.deviceIdentities,
]);
let device = await txn.deviceIdentities.get(userId, deviceId);
if (device) {
log.set("existingDevice", true);
} else {
//// BEGIN EXTRACT (deviceKeysMap)
const deviceKeyResponse = await hsApi.queryKeys({
"timeout": 10000,
"device_keys": {
[userId]: [deviceId]
},
"token": this._getSyncToken()
}, {log}).response();
// verify signature
const verifiedKeysPerUser = log.wrap("verify", log => this._filterVerifiedDeviceKeys(deviceKeyResponse["device_keys"], log));
//// END EXTRACT
// TODO: what if verifiedKeysPerUser is empty or does not contain userId?
const verifiedKeys = verifiedKeysPerUser
.find(vkpu => vkpu.userId === userId).verifiedKeys
.find(vk => vk["device_id"] === deviceId);
// user hasn't uploaded keys for device?
if (!verifiedKeys) {
return undefined;
}
device = deviceKeysAsDeviceIdentity(verifiedKeys);
const txn = await this._storage.readWriteTxn([
this._storage.storeNames.deviceIdentities,
]);
// check again we don't have the device already.
// when updating all keys for a user we allow updating the
// device when the key hasn't changed so the device display name
// can be updated, but here we don't.
const existingDevice = await txn.deviceIdentities.get(userId, deviceId);
if (existingDevice) {
device = existingDevice;
log.set("existingDeviceAfterFetch", true);
} else {
try {
txn.deviceIdentities.set(device);
log.set("newDevice", true);
} catch (err) {
txn.abort();
throw err;
}
await txn.complete();
}
}
return device;
}
/**
* Gets all the device identities with which keys should be shared for a set of users in a tracked room.
* If any userIdentities are outdated, it will fetch them from the homeserver.

3
src/matrix/e2ee/README.md
View File

@ -41,5 +41,8 @@ Runs before any room.prepareSync, so the new room keys can be passed to each roo
- e2ee account
- generate more otks if needed
- upload new otks if needed or device keys if not uploaded before
- device message handler:
- fetch keys we don't know about yet for (call) to_device messages identity
- pass signalling messages to call handler
- rooms
- share new room keys if needed

14
src/matrix/e2ee/RoomEncryption.js
View File

@ -18,7 +18,7 @@ import {MEGOLM_ALGORITHM, DecryptionSource} from "./common.js";
import {groupEventsBySession} from "./megolm/decryption/utils";
import {mergeMap} from "../../utils/mergeMap";
import {groupBy} from "../../utils/groupBy";
import {makeTxnId} from "../common.js";
import {makeTxnId, formatToDeviceMessagesPayload} from "../common.js";
import {iterateResponseStateEvents} from "../room/common";
const ENCRYPTED_TYPE = "m.room.encrypted";
@ -457,6 +457,7 @@ export class RoomEncryption {
await writeTxn.complete();
}
// TODO: make this use _sendMessagesToDevices
async _sendSharedMessageToDevices(type, message, devices, hsApi, log) {
const devicesByUser = groupBy(devices, device => device.userId);
const payload = {
@ -474,16 +475,7 @@ export class RoomEncryption {
async _sendMessagesToDevices(type, messages, hsApi, log) {
log.set("messages", messages.length);
const messagesByUser = groupBy(messages, message => message.device.userId);
const payload = {
messages: Array.from(messagesByUser.entries()).reduce((userMap, [userId, messages]) => {
userMap[userId] = messages.reduce((deviceMap, message) => {
deviceMap[message.device.deviceId] = message.content;
return deviceMap;
}, {});
return userMap;
}, {})
};
const payload = formatToDeviceMessagesPayload(messages);
const txnId = makeTxnId();
await hsApi.sendToDevice(type, payload, txnId, {log}).response();
}

2
src/matrix/e2ee/megolm/keybackup/KeyBackup.ts
View File

@ -19,7 +19,7 @@ import {StoredRoomKey, keyFromBackup} from "../decryption/RoomKey";
import {MEGOLM_ALGORITHM} from "../../common";
import * as Curve25519 from "./Curve25519";
import {AbortableOperation} from "../../../../utils/AbortableOperation";
import {ObservableValue} from "../../../../observable/ObservableValue";
import {ObservableValue} from "../../../../observable/value";
import {SetAbortableFn} from "../../../../utils/AbortableOperation";
import type {BackupInfo, SessionData, SessionKeyInfo, SessionInfo, KeyBackupPayload} from "./types";

40
src/matrix/e2ee/olm/Encryption.ts
View File

@ -21,6 +21,7 @@ import {createSessionEntry} from "./Session";
import type {OlmMessage, OlmPayload, OlmEncryptedMessageContent} from "./types";
import type {Account} from "../Account";
import type {LockMap} from "../../../utils/LockMap";
import {Lock, MultiLock, ILock} from "../../../utils/Lock";
import type {Storage} from "../../storage/idb/Storage";
import type {Transaction} from "../../storage/idb/Transaction";
import type {DeviceIdentity} from "../../storage/idb/stores/DeviceIdentityStore";
@ -62,6 +63,9 @@ const OTK_ALGORITHM = "signed_curve25519";
const MAX_BATCH_SIZE = 20;
export class Encryption {
private _batchLocks: Array<Lock>;
constructor(
private readonly account: Account,
private readonly pickleKey: string,
@ -71,14 +75,42 @@ export class Encryption {
private readonly ownUserId: string,
private readonly olmUtil: Olm.Utility,
private readonly senderKeyLock: LockMap<string>
) {}
) {
this._batchLocks = new Array(MAX_BATCH_SIZE);
for (let i = 0; i < MAX_BATCH_SIZE; i += 1) {
this._batchLocks[i] = new Lock();
}
}
/** A hack to prevent olm OOMing when `encrypt` is called several times concurrently,
* which is the case when encrypting voip signalling message to send over to_device.
* A better fix will be to extract the common bits from megolm/KeyLoader in a super class
* and have some sort of olm/SessionLoader that is shared between encryption and decryption
* and only keeps the olm session in wasm memory for a brief moment, like we already do for RoomKeys,
* and get the benefit of an optimal cache at the same time.
* */
private async _takeBatchLock(amount: number): Promise<ILock> {
const locks = this._batchLocks.filter(l => !l.isTaken).slice(0, amount);
if (locks.length < amount) {
const takenLocks = this._batchLocks.filter(l => l.isTaken).slice(0, amount - locks.length);
locks.push(...takenLocks);
}
await Promise.all(locks.map(l => l.take()));
return new MultiLock(locks);
}
async encrypt(type: string, content: Record<string, any>, devices: DeviceIdentity[], hsApi: HomeServerApi, log: ILogItem): Promise<EncryptedMessage[]> {
let messages: EncryptedMessage[] = [];
for (let i = 0; i < devices.length ; i += MAX_BATCH_SIZE) {
const batchDevices = devices.slice(i, i + MAX_BATCH_SIZE);
const batchMessages = await this._encryptForMaxDevices(type, content, batchDevices, hsApi, log);
messages = messages.concat(batchMessages);
const batchLock = await this._takeBatchLock(batchDevices.length);
try {
const batchMessages = await this._encryptForMaxDevices(type, content, batchDevices, hsApi, log);
messages = messages.concat(batchMessages);
}
finally {
batchLock.release();
}
}
return messages;
}
@ -311,7 +343,7 @@ class EncryptionTarget {
}
}
class EncryptedMessage {
export class EncryptedMessage {
constructor(
public readonly content: OlmEncryptedMessageContent,
public readonly device: DeviceIdentity

10
src/matrix/net/HomeServerApi.ts
View File

@ -159,6 +159,10 @@ export class HomeServerApi {
state(roomId: string, eventType: string, stateKey: string, options?: BaseRequestOptions): IHomeServerRequest {
return this._get(`/rooms/${encodeURIComponent(roomId)}/state/${encodeURIComponent(eventType)}/${encodeURIComponent(stateKey)}`, {}, undefined, options);
}
sendState(roomId: string, eventType: string, stateKey: string, content: Record<string, any>, options?: BaseRequestOptions): IHomeServerRequest {
return this._put(`/rooms/${encodeURIComponent(roomId)}/state/${encodeURIComponent(eventType)}/${encodeURIComponent(stateKey)}`, {}, content, options);
}
getLoginFlows(): IHomeServerRequest {
return this._unauthedRequest("GET", this._url("/login"));
@ -301,10 +305,14 @@ export class HomeServerApi {
createRoom(payload: Record<string, any>, options?: BaseRequestOptions): IHomeServerRequest {
return this._post(`/createRoom`, {}, payload, options);
}
setAccountData(ownUserId: string, type: string, content: Record<string, any>, options?: BaseRequestOptions): IHomeServerRequest {
return this._put(`/user/${encodeURIComponent(ownUserId)}/account_data/${encodeURIComponent(type)}`, {}, content, options);
}
getTurnServer(options?: BaseRequestOptions): IHomeServerRequest {
return this._get(`/voip/turnServer`, undefined, undefined, options);
}
}
import {Request as MockRequest} from "../../mocks/Request.js";

13
src/matrix/net/MediaRepository.ts
View File

@ -29,32 +29,31 @@ export class MediaRepository {
this._platform = platform;
}
mxcUrlThumbnail(url: string, width: number, height: number, method: "crop" | "scale"): string | null {
mxcUrlThumbnail(url: string, width: number, height: number, method: "crop" | "scale"): string | undefined {
const parts = this._parseMxcUrl(url);
if (parts) {
const [serverName, mediaId] = parts;
const httpUrl = `${this._homeserver}/_matrix/media/r0/thumbnail/${encodeURIComponent(serverName)}/${encodeURIComponent(mediaId)}`;
return httpUrl + "?" + encodeQueryParams({width: Math.round(width), height: Math.round(height), method});
}
return null;
return undefined;
}
mxcUrl(url: string): string | null {
mxcUrl(url: string): string | undefined {
const parts = this._parseMxcUrl(url);
if (parts) {
const [serverName, mediaId] = parts;
return `${this._homeserver}/_matrix/media/r0/download/${encodeURIComponent(serverName)}/${encodeURIComponent(mediaId)}`;
} else {
return null;
}
return undefined;
}
private _parseMxcUrl(url: string): string[] | null {
private _parseMxcUrl(url: string): string[] | undefined {
const prefix = "mxc://";
if (url.startsWith(prefix)) {
return url.substr(prefix.length).split("/", 2);
} else {
return null;
return undefined;
}
}

2
src/matrix/net/Reconnector.ts
View File

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableValue} from "../../observable/ObservableValue";
import {ObservableValue} from "../../observable/value";
import type {ExponentialRetryDelay} from "./ExponentialRetryDelay";
import type {TimeMeasure} from "../../platform/web/dom/Clock.js";
import type {OnlineStatus} from "../../platform/web/dom/OnlineStatus.js";

8
src/matrix/room/ArchivedRoom.js
View File

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {reduceStateEvents} from "./RoomSummary.js";
import {iterateResponseStateEvents} from "./common";
import {BaseRoom} from "./BaseRoom.js";
import {RoomMember, EVENT_TYPE as MEMBER_EVENT_TYPE} from "./members/RoomMember.js";
@ -173,15 +173,15 @@ export class ArchivedRoom extends BaseRoom {
}
function findKickDetails(roomResponse, ownUserId) {
const kickEvent = reduceStateEvents(roomResponse, (kickEvent, event) => {
let kickEvent;
iterateResponseStateEvents(roomResponse, event => {
if (event.type === MEMBER_EVENT_TYPE) {
// did we get kicked?
if (event.state_key === ownUserId && event.sender !== event.state_key) {
kickEvent = event;
}
}
return kickEvent;
}, null);
});
if (kickEvent) {
return {
// this is different from the room membership in the sync section, which can only be leave

32
src/matrix/room/BaseRoom.js
View File

@ -29,8 +29,10 @@ import {ObservedEventMap} from "./ObservedEventMap.js";
import {DecryptionSource} from "../e2ee/common.js";
import {ensureLogItem} from "../../logging/utils";
import {PowerLevels} from "./PowerLevels.js";
import {RetainedObservableValue} from "../../observable/ObservableValue";
import {RetainedObservableValue} from "../../observable/value";
import {TimelineReader} from "./timeline/persistence/TimelineReader";
import {ObservedStateTypeMap} from "./state/ObservedStateTypeMap";
import {ObservedStateKeyValue} from "./state/ObservedStateKeyValue";
const EVENT_ENCRYPTED_TYPE = "m.room.encrypted";
@ -53,11 +55,35 @@ export class BaseRoom extends EventEmitter {
this._getSyncToken = getSyncToken;
this._platform = platform;
this._observedEvents = null;
this._roomStateObservers = new Set();
this._powerLevels = null;
this._powerLevelLoading = null;
this._observedMembers = null;
}
async observeStateType(type, txn = undefined) {
const map = new ObservedStateTypeMap(type);
await this._addStateObserver(map, txn);
return map;
}
async observeStateTypeAndKey(type, stateKey, txn = undefined) {
const value = new ObservedStateKeyValue(type, stateKey);
await this._addStateObserver(value, txn);
return value;
}
async _addStateObserver(stateObserver, txn) {
if (!txn) {
txn = await this._storage.readTxn([this._storage.storeNames.roomState]);
}
await stateObserver.load(this.id, txn);
this._roomStateObservers.add(stateObserver);
stateObserver.setRemoveCallback(() => {
this._roomStateObservers.delete(stateObserver);
});
}
async _eventIdsToEntries(eventIds, txn) {
const retryEntries = [];
await Promise.all(eventIds.map(async eventId => {
@ -433,6 +459,10 @@ export class BaseRoom extends EventEmitter {
return this._summary.data.membership;
}
get user() {
return this._user;
}
isDirectMessageForUserId(userId) {
if (this._summary.data.dmUserId === userId) {
return true;

2
src/matrix/room/ObservedEventMap.js
View File

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "../../observable/ObservableValue";
import {BaseObservableValue} from "../../observable/value";
export class ObservedEventMap {
constructor(notifyEmpty) {

37
src/matrix/room/Room.js
View File

@ -23,6 +23,7 @@ import {WrappedError} from "../error.js"
import {Heroes} from "./members/Heroes.js";
import {AttachmentUpload} from "./AttachmentUpload.js";
import {DecryptionSource} from "../e2ee/common.js";
import {iterateResponseStateEvents} from "./common";
import {PowerLevels, EVENT_TYPE as POWERLEVELS_EVENT_TYPE } from "./PowerLevels.js";
const EVENT_ENCRYPTED_TYPE = "m.room.encrypted";
@ -30,6 +31,7 @@ const EVENT_ENCRYPTED_TYPE = "m.room.encrypted";
export class Room extends BaseRoom {
constructor(options) {
super(options);
this._roomStateHandler = options.roomStateHandler;
// TODO: pass pendingEvents to start like pendingOperations?
const {pendingEvents} = options;
const relationWriter = new RelationWriter({
@ -121,7 +123,7 @@ export class Room extends BaseRoom {
txn.roomState.removeAllForRoom(this.id);
txn.roomMembers.removeAllForRoom(this.id);
}
const {entries: newEntries, updatedEntries, newLiveKey, memberChanges} =
const {entries: newEntries, updatedEntries, newLiveKey, memberChanges, memberSync} =
await log.wrap("syncWriter", log => this._syncWriter.writeSync(
roomResponse, isRejoin, summaryChanges.hasFetchedMembers, txn, log), log.level.Detail);
let decryption;
@ -179,7 +181,9 @@ export class Room extends BaseRoom {
removedPendingEvents = await this._sendQueue.removeRemoteEchos(roomResponse.timeline.events, txn, log);
}
const powerLevelsEvent = this._getPowerLevelsEvent(roomResponse);
await this._runRoomStateHandlers(roomResponse, memberSync, txn, log);
return {
roomResponse,
summaryChanges,
roomEncryption,
newEntries,
@ -203,7 +207,7 @@ export class Room extends BaseRoom {
const {
summaryChanges, newEntries, updatedEntries, newLiveKey,
removedPendingEvents, memberChanges, powerLevelsEvent,
heroChanges, roomEncryption, encryptionChanges
heroChanges, roomEncryption, roomResponse, encryptionChanges
} = changes;
log.set("id", this.id);
this._syncWriter.afterSync(newLiveKey);
@ -220,6 +224,7 @@ export class Room extends BaseRoom {
if (this._memberList) {
this._memberList.afterSync(memberChanges);
}
this._roomStateHandler.updateRoomMembers(this, memberChanges);
if (this._observedMembers) {
this._updateObservedMembers(memberChanges);
}
@ -265,6 +270,7 @@ export class Room extends BaseRoom {
if (removedPendingEvents) {
this._sendQueue.emitRemovals(removedPendingEvents);
}
this._emitSyncRoomState(roomResponse);
}
_updateObservedMembers(memberChanges) {
@ -277,8 +283,13 @@ export class Room extends BaseRoom {
}
_getPowerLevelsEvent(roomResponse) {
const isPowerlevelEvent = event => event.state_key === "" && event.type === POWERLEVELS_EVENT_TYPE;
const powerLevelEvent = roomResponse.timeline?.events.find(isPowerlevelEvent) ?? roomResponse.state?.events.find(isPowerlevelEvent);
let powerLevelEvent;
iterateResponseStateEvents(roomResponse, event => {
if(event.state_key === "" && event.type === POWERLEVELS_EVENT_TYPE) {
powerLevelEvent = event;
}
});
return powerLevelEvent;
}
@ -464,6 +475,24 @@ export class Room extends BaseRoom {
return this._sendQueue.pendingEvents;
}
/** global room state handlers, run during writeSync step */
_runRoomStateHandlers(roomResponse, memberSync, txn, log) {
const promises = [];
iterateResponseStateEvents(roomResponse, event => {
promises.push(this._roomStateHandler.handleRoomState(this, event, memberSync, txn, log));
});
return Promise.all(promises);
}
/** local room state observers, run during afterSync step */
_emitSyncRoomState(roomResponse) {
iterateResponseStateEvents(roomResponse, event => {
for (const handler of this._roomStateObservers) {
handler.handleStateEvent(event);
}
});
}
/** @package */
writeIsTrackingMembers(value, txn) {
return this._summary.writeIsTrackingMembers(value, txn);

25
src/matrix/room/RoomSummary.js
View File

@ -15,7 +15,7 @@ limitations under the License.
*/
import {MEGOLM_ALGORITHM} from "../e2ee/common.js";
import {iterateResponseStateEvents} from "./common";
function applyTimelineEntries(data, timelineEntries, isInitialSync, canMarkUnread, ownUserId) {
if (timelineEntries.length) {
@ -27,25 +27,6 @@ function applyTimelineEntries(data, timelineEntries, isInitialSync, canMarkUnrea
return data;
}
export function reduceStateEvents(roomResponse, callback, value) {
const stateEvents = roomResponse?.state?.events;
// state comes before timeline
if (Array.isArray(stateEvents)) {
value = stateEvents.reduce(callback, value);
}
const timelineEvents = roomResponse?.timeline?.events;
// and after that state events in the timeline
if (Array.isArray(timelineEvents)) {
value = timelineEvents.reduce((data, event) => {
if (typeof event.state_key === "string") {
value = callback(value, event);
}
return value;
}, value);
}
return value;
}
function applySyncResponse(data, roomResponse, membership, ownUserId) {
if (roomResponse.summary) {
data = updateSummary(data, roomResponse.summary);
@ -60,7 +41,9 @@ function applySyncResponse(data, roomResponse, membership, ownUserId) {
// process state events in state and in timeline.
// non-state events are handled by applyTimelineEntries
// so decryption is handled properly
data = reduceStateEvents(roomResponse, (data, event) => processStateEvent(data, event, ownUserId), data);
iterateResponseStateEvents(roomResponse, event => {
data = processStateEvent(data, event, ownUserId);
});
const unreadNotifications = roomResponse.unread_notifications;
if (unreadNotifications) {
data = processNotificationCounts(data, unreadNotifications);

1
src/matrix/room/common.ts
View File

@ -53,6 +53,7 @@ type RoomResponse = {
}
/** iterates over any state events in a sync room response, in the order that they should be applied (from older to younger events) */
export function iterateResponseStateEvents(roomResponse: RoomResponse, callback: (StateEvent) => Promise<void> | void): Promise<void> | void {
let promises: Promise<void>[] | undefined = undefined;
const callCallback = stateEvent => {

104
src/matrix/room/state/ObservedStateKeyValue.ts Normal file
View File

@ -0,0 +1,104 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {StateObserver} from "./types";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import {BaseObservableValue} from "../../../observable/value";
/**
* Observable value for a state event with a given type and state key.
* Unsubscribes when last subscription is removed */
export class ObservedStateKeyValue extends BaseObservableValue<StateEvent | undefined> implements StateObserver {
private event?: StateEvent;
private removeCallback?: () => void;
constructor(private readonly type: string, private readonly stateKey: string) {
super();
}
/** @internal */
async load(roomId: string, txn: Transaction): Promise<void> {
this.event = (await txn.roomState.get(roomId, this.type, this.stateKey))?.event;
}
/** @internal */
handleStateEvent(event: StateEvent) {
if (event.type === this.type && event.state_key === this.stateKey) {
this.event = event;
this.emit(this.get());
}
}
get(): StateEvent | undefined {
return this.event;
}
setRemoveCallback(callback: () => void) {
this.removeCallback = callback;
}
onUnsubscribeLast() {
this.removeCallback?.();
}
}
import {createMockStorage} from "../../../mocks/Storage";
export async function tests() {
return {
"test load and update": async assert => {
const storage = await createMockStorage();
const writeTxn = await storage.readWriteTxn([storage.storeNames.roomState]);
writeTxn.roomState.set("!abc", {
event_id: "$abc",
type: "m.room.member",
state_key: "@alice",
sender: "@alice",
origin_server_ts: 5,
content: {}
});
await writeTxn.complete();
const txn = await storage.readTxn([storage.storeNames.roomState]);
const value = new ObservedStateKeyValue("m.room.member", "@alice");
await value.load("!abc", txn);
const updates: Array<StateEvent | undefined> = [];
assert.strictEqual(value.get()?.origin_server_ts, 5);
const unsubscribe = value.subscribe(value => updates.push(value));
value.handleStateEvent({
event_id: "$abc",
type: "m.room.member",
state_key: "@bob",
sender: "@alice",
origin_server_ts: 10,
content: {}
});
assert.strictEqual(updates.length, 0);
value.handleStateEvent({
event_id: "$abc",
type: "m.room.member",
state_key: "@alice",
sender: "@alice",
origin_server_ts: 10,
content: {}
});
assert.strictEqual(updates.length, 1);
assert.strictEqual(updates[0]?.origin_server_ts, 10);
let removed = false;
value.setRemoveCallback(() => removed = true);
unsubscribe();
assert(removed);
}
}
}

53
src/matrix/room/state/ObservedStateTypeMap.ts Normal file
View File

@ -0,0 +1,53 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {StateObserver} from "./types";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import {ObservableMap} from "../../../observable/map";
/**
* Observable map for a given type with state keys as map keys.
* Unsubscribes when last subscription is removed */
export class ObservedStateTypeMap extends ObservableMap<string, StateEvent> implements StateObserver {
private removeCallback?: () => void;
constructor(private readonly type: string) {
super();
}
/** @internal */
async load(roomId: string, txn: Transaction): Promise<void> {
const events = await txn.roomState.getAllForType(roomId, this.type);
for (let i = 0; i < events.length; ++i) {
const {event} = events[i];
this.add(event.state_key, event);
}
}
/** @internal */
handleStateEvent(event: StateEvent) {
if (event.type === this.type) {
this.set(event.state_key, event);
}
}
setRemoveCallback(callback: () => void) {
this.removeCallback = callback;
}
onUnsubscribeLast() {
this.removeCallback?.();
}
}

40
src/matrix/room/state/RoomStateHandlerSet.ts Normal file
View File

@ -0,0 +1,40 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {ILogItem} from "../../../logging/types";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import type {Room} from "../Room";
import type {MemberChange} from "../members/RoomMember";
import type {RoomStateHandler} from "./types";
import type {MemberSync} from "../timeline/persistence/MemberWriter.js";
import {BaseObservable} from "../../../observable/BaseObservable";
/** keeps track of all handlers registered with Session.observeRoomState */
export class RoomStateHandlerSet extends BaseObservable<RoomStateHandler> implements RoomStateHandler {
async handleRoomState(room: Room, stateEvent: StateEvent, memberSync: MemberSync, txn: Transaction, log: ILogItem): Promise<void> {
const promises: Promise<void>[] = [];
for(let h of this._handlers) {
promises.push(h.handleRoomState(room, stateEvent, memberSync, txn, log));
}
await Promise.all(promises);
}
updateRoomMembers(room: Room, memberChanges: Map<string, MemberChange>) {
for(let h of this._handlers) {
h.updateRoomMembers(room, memberChanges);
}
}
}

39
src/matrix/room/state/types.ts Normal file
View File

@ -0,0 +1,39 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import type {Room} from "../Room";
import type {StateEvent} from "../../storage/types";
import type {Transaction} from "../../storage/idb/Transaction";
import type {ILogItem} from "../../../logging/types";
import type {MemberChange} from "../members/RoomMember";
import type {MemberSync} from "../timeline/persistence/MemberWriter";
/** used for Session.observeRoomState, which observes in all room, but without loading from storage
* It receives the sync write transaction, so other stores can be updated as part of the same transaction. */
export interface RoomStateHandler {
handleRoomState(room: Room, stateEvent: StateEvent, memberSync: MemberSync, syncWriteTxn: Transaction, log: ILogItem): Promise<void>;
updateRoomMembers(room: Room, memberChanges: Map<string, MemberChange>): void;
}
/**
* used for Room.observeStateType and Room.observeStateTypeAndKey
* @internal
* */
export interface StateObserver {
handleStateEvent(event: StateEvent);
load(roomId: string, txn: Transaction): Promise<void>;
setRemoveCallback(callback: () => void);
}

2
src/matrix/room/timeline/entries/BaseEventEntry.js
View File

@ -181,7 +181,7 @@ export class BaseEventEntry extends BaseEntry {
return createAnnotation(this.id, key);
}
reply(msgtype, body) {
createReplyContent(msgtype, body) {
return createReplyContent(this, msgtype, body);
}

6
src/matrix/room/timeline/persistence/MemberWriter.js
View File

@ -56,7 +56,11 @@ export class MemberWriter {
}
}
class MemberSync {
/** Represents the member changes in a given sync.
* Used to write the changes to storage and historical member
* information for events in the same sync.
**/
export class MemberSync {
constructor(memberWriter, stateEvents, timelineEvents, hasFetchedMembers) {
this._memberWriter = memberWriter;
this._timelineEvents = timelineEvents;

2
src/matrix/room/timeline/persistence/SyncWriter.js
View File

@ -244,7 +244,7 @@ export class SyncWriter {
const {currentKey, entries, updatedEntries} =
await this._writeTimeline(timelineEvents, timeline, memberSync, this._lastLiveKey, txn, log);
const memberChanges = await memberSync.write(txn);
return {entries, updatedEntries, newLiveKey: currentKey, memberChanges};
return {entries, updatedEntries, newLiveKey: currentKey, memberChanges, memberSync};
}
afterSync(newLiveKey) {

1
src/matrix/storage/common.ts
View File

@ -33,6 +33,7 @@ export enum StoreNames {
groupSessionDecryptions = "groupSessionDecryptions",
operations = "operations",
accountData = "accountData",
calls = "calls"
}
export const STORE_NAMES: Readonly<StoreNames[]> = Object.values(StoreNames);

2
src/matrix/storage/idb/StorageFactory.ts
View File

@ -69,7 +69,7 @@ export class StorageFactory {
requestPersistedStorage().then(persisted => {
// Firefox lies here though, and returns true even if the user denied the request
if (!persisted) {
console.warn("no persisted storage, database can be evicted by browser");
log.log("no persisted storage, database can be evicted by browser", log.level.Warn);
}
});

5
src/matrix/storage/idb/Transaction.ts
View File

@ -36,6 +36,7 @@ import {OutboundGroupSessionStore} from "./stores/OutboundGroupSessionStore";
import {GroupSessionDecryptionStore} from "./stores/GroupSessionDecryptionStore";
import {OperationStore} from "./stores/OperationStore";
import {AccountDataStore} from "./stores/AccountDataStore";
import {CallStore} from "./stores/CallStore";
import type {ILogger, ILogItem} from "../../../logging/types";
export type IDBKey = IDBValidKey | IDBKeyRange;
@ -167,6 +168,10 @@ export class Transaction {
get accountData(): AccountDataStore {
return this._store(StoreNames.accountData, idbStore => new AccountDataStore(idbStore));
}
get calls(): CallStore {
return this._store(StoreNames.calls, idbStore => new CallStore(idbStore));
}
async complete(log?: ILogItem): Promise<void> {
try {

8
src/matrix/storage/idb/schema.ts
View File

@ -33,7 +33,8 @@ export const schema: MigrationFunc[] = [
backupAndRestoreE2EEAccountToLocalStorage,
clearAllStores,
addInboundSessionBackupIndex,
migrateBackupStatus
migrateBackupStatus,
createCallStore
];
// TODO: how to deal with git merge conflicts of this array?
@ -269,3 +270,8 @@ async function migrateBackupStatus(db: IDBDatabase, txn: IDBTransaction, localSt
log.set("countWithoutSession", countWithoutSession);
log.set("countWithSession", countWithSession);
}
//v17 create calls store
function createCallStore(db: IDBDatabase) : void {
db.createObjectStore("calls", {keyPath: "key"});
}

83
src/matrix/storage/idb/stores/CallStore.ts Normal file
View File

@ -0,0 +1,83 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Copyright 2021 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {Store} from "../Store";
import {StateEvent} from "../../types";
import {MIN_UNICODE, MAX_UNICODE} from "./common";
function encodeKey(intent: string, roomId: string, callId: string) {
return `${intent}|${roomId}|${callId}`;
}
function decodeStorageEntry(storageEntry: CallStorageEntry): CallEntry {
const [intent, roomId, callId] = storageEntry.key.split("|");
return {intent, roomId, callId, timestamp: storageEntry.timestamp};
}
export interface CallEntry {
intent: string;
roomId: string;
callId: string;
timestamp: number;
}
type CallStorageEntry = {
key: string;
timestamp: number;
}
export class CallStore {
private _callStore: Store<CallStorageEntry>;
constructor(idbStore: Store<CallStorageEntry>) {
this._callStore = idbStore;
}
async getByIntent(intent: string): Promise<CallEntry[]> {
const range = this._callStore.IDBKeyRange.bound(
encodeKey(intent, MIN_UNICODE, MIN_UNICODE),
encodeKey(intent, MAX_UNICODE, MAX_UNICODE),
true,
true
);
const storageEntries = await this._callStore.selectAll(range);
return storageEntries.map(e => decodeStorageEntry(e));
}
async getByIntentAndRoom(intent: string, roomId: string): Promise<CallEntry[]> {
const range = this._callStore.IDBKeyRange.bound(
encodeKey(intent, roomId, MIN_UNICODE),
encodeKey(intent, roomId, MAX_UNICODE),
true,
true
);
const storageEntries = await this._callStore.selectAll(range);
return storageEntries.map(e => decodeStorageEntry(e));
}
add(entry: CallEntry) {
const storageEntry: CallStorageEntry = {
key: encodeKey(entry.intent, entry.roomId, entry.callId),
timestamp: entry.timestamp
};
this._callStore.add(storageEntry);
}
remove(intent: string, roomId: string, callId: string): void {
this._callStore.delete(encodeKey(intent, roomId, callId));
}
}

12
src/matrix/storage/idb/stores/RoomStateStore.ts
View File

@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {MAX_UNICODE} from "./common";
import {MIN_UNICODE, MAX_UNICODE} from "./common";
import {Store} from "../Store";
import {StateEvent} from "../../types";
@ -41,6 +41,16 @@ export class RoomStateStore {
return this._roomStateStore.get(key);
}
getAllForType(roomId: string, type: string): Promise<RoomStateEntry[]> {
const range = this._roomStateStore.IDBKeyRange.bound(
encodeKey(roomId, type, ""),
encodeKey(roomId, type, MAX_UNICODE),
false,
true
);
return this._roomStateStore.selectAll(range);
}
set(roomId: string, event: StateEvent): void {
const key = encodeKey(roomId, event.type, event.state_key);
const entry = {roomId, event, key};

2
src/mocks/Clock.js
View File

@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableValue} from "../observable/ObservableValue";
import {ObservableValue} from "../observable/value";
class Timeout {
constructor(elapsed, ms) {

249
src/observable/ObservableValue.ts
View File

@ -1,249 +0,0 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AbortError} from "../utils/error";
import {BaseObservable} from "./BaseObservable";
import type {SubscriptionHandle} from "./BaseObservable";
// like an EventEmitter, but doesn't have an event type
export abstract class BaseObservableValue<T> extends BaseObservable<(value: T) => void> {
emit(argument: T): void {
for (const h of this._handlers) {
h(argument);
}
}
abstract get(): T;
waitFor(predicate: (value: T) => boolean): IWaitHandle<T> {
if (predicate(this.get())) {
return new ResolvedWaitForHandle(Promise.resolve(this.get()));
} else {
return new WaitForHandle(this, predicate);
}
}
flatMap<C>(mapper: (value: T) => (BaseObservableValue<C> | undefined)): BaseObservableValue<C | undefined> {
return new FlatMapObservableValue<T, C>(this, mapper);
}
}
interface IWaitHandle<T> {
promise: Promise<T>;
dispose(): void;
}
class WaitForHandle<T> implements IWaitHandle<T> {
private _promise: Promise<T>
private _reject: ((reason?: any) => void) | null;
private _subscription: (() => void) | null;
constructor(observable: BaseObservableValue<T>, predicate: (value: T) => boolean) {
this._promise = new Promise((resolve, reject) => {
this._reject = reject;
this._subscription = observable.subscribe(v => {
if (predicate(v)) {
this._reject = null;
resolve(v);
this.dispose();
}
});
});
}
get promise(): Promise<T> {
return this._promise;
}
dispose(): void {
if (this._subscription) {
this._subscription();
this._subscription = null;
}
if (this._reject) {
this._reject(new AbortError());
this._reject = null;
}
}
}
class ResolvedWaitForHandle<T> implements IWaitHandle<T> {
constructor(public promise: Promise<T>) {}
dispose(): void {}
}
export class ObservableValue<T> extends BaseObservableValue<T> {
private _value: T;
constructor(initialValue: T) {
super();
this._value = initialValue;
}
get(): T {
return this._value;
}
set(value: T): void {
if (value !== this._value) {
this._value = value;
this.emit(this._value);
}
}
}
export class RetainedObservableValue<T> extends ObservableValue<T> {
private _freeCallback: () => void;
constructor(initialValue: T, freeCallback: () => void) {
super(initialValue);
this._freeCallback = freeCallback;
}
onUnsubscribeLast(): void {
super.onUnsubscribeLast();
this._freeCallback();
}
}
export class FlatMapObservableValue<P, C> extends BaseObservableValue<C | undefined> {
private sourceSubscription?: SubscriptionHandle;
private targetSubscription?: SubscriptionHandle;
constructor(
private readonly source: BaseObservableValue<P>,
private readonly mapper: (value: P) => (BaseObservableValue<C> | undefined)
) {
super();
}
onUnsubscribeLast(): void {
super.onUnsubscribeLast();
this.sourceSubscription = this.sourceSubscription!();
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
onSubscribeFirst(): void {
super.onSubscribeFirst();
this.sourceSubscription = this.source.subscribe(() => {
this.updateTargetSubscription();
this.emit(this.get());
});
this.updateTargetSubscription();
}
private updateTargetSubscription(): void {
const sourceValue = this.source.get();
if (sourceValue) {
const target = this.mapper(sourceValue);
if (target) {
if (!this.targetSubscription) {
this.targetSubscription = target.subscribe(() => this.emit(this.get()));
}
return;
}
}
// if no sourceValue or target
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
get(): C | undefined {
const sourceValue = this.source.get();
if (!sourceValue) {
return undefined;
}
const mapped = this.mapper(sourceValue);
return mapped?.get();
}
}
// eslint-disable-next-line @typescript-eslint/explicit-function-return-type
export function tests() {
return {
"set emits an update": (assert): void => {
const a = new ObservableValue<number>(0);
let fired = false;
const subscription = a.subscribe(v => {
fired = true;
assert.strictEqual(v, 5);
});
a.set(5);
assert(fired);
subscription();
},
"set doesn't emit if value hasn't changed": (assert): void => {
const a = new ObservableValue(5);
let fired = false;
const subscription = a.subscribe(() => {
fired = true;
});
a.set(5);
a.set(5);
assert(!fired);
subscription();
},
"waitFor promise resolves on matching update": async (assert): Promise<void> => {
const a = new ObservableValue(5);
const handle = a.waitFor(v => v === 6);
await Promise.resolve().then(() => {
a.set(6);
});
await handle.promise;
assert.strictEqual(a.get(), 6);
},
"waitFor promise rejects when disposed": async (assert): Promise<void> => {
const a = new ObservableValue<number>(0);
const handle = a.waitFor(() => false);
await Promise.resolve().then(() => {
handle.dispose();
});
await assert.rejects(handle.promise, AbortError);
},
"flatMap.get": (assert): void => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const countProxy = a.flatMap(a => a!.count);
assert.strictEqual(countProxy.get(), undefined);
const count = new ObservableValue<number>(0);
a.set({count});
assert.strictEqual(countProxy.get(), 0);
},
"flatMap update from source": (assert): void => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
a.flatMap(a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
assert.deepEqual(updates, [0]);
},
"flatMap update from target": (assert): void => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
a.flatMap(a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
count.set(5);
assert.deepEqual(updates, [0, 5]);
}
};
}

6
src/observable/index.ts
View File

@ -1,12 +1,10 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@ -21,4 +19,4 @@ export { ObservableArray } from "./list/ObservableArray";
export { SortedArray } from "./list/SortedArray";
export { MappedList } from "./list/MappedList";
export { AsyncMappedList } from "./list/AsyncMappedList";
export { ConcatList } from "./list/ConcatList";
export { ConcatList } from "./list/ConcatList";

27
src/observable/map/BaseObservableMap.ts
View File

@ -18,6 +18,7 @@ import {BaseObservable} from "../BaseObservable";
import {JoinedMap} from "./index";
import {MappedMap} from "./index";
import {FilteredMap} from "./index";
import {BaseObservableValue, MapSizeObservableValue} from "../value/index";
import {SortedMapList} from "../list/SortedMapList.js";
@ -66,19 +67,23 @@ export abstract class BaseObservableMap<K, V> extends BaseObservable<IMapObserve
join<E extends BaseObservableMap<K, V>>(...otherMaps: Array<E>): JoinedMap<K, V> {
return new JoinedMap([this as BaseObservableMap<K, V>].concat(otherMaps));
}
}
mapValues<MappedV>(mapper: Mapper<V, MappedV>, updater?: Updater<V, MappedV>): MappedMap<K, V, MappedV> {
return new MappedMap(this, mapper, updater);
}
mapValues<MappedV>(mapper: Mapper<V, MappedV>, updater?: Updater<V, MappedV>): MappedMap<K, V, MappedV> {
return new MappedMap(this, mapper, updater);
}
sortValues(comparator: Comparator<V>): SortedMapList {
return new SortedMapList(this, comparator);
}
sortValues(comparator: Comparator<V>): SortedMapList {
return new SortedMapList(this, comparator);
}
filterValues(filter: Filter<K, V>): FilteredMap<K, V> {
return new FilteredMap(this, filter);
}
filterValues(filter: Filter<K, V>): FilteredMap<K, V> {
return new FilteredMap(this, filter);
}
observeSize(): BaseObservableValue<number> {
return new MapSizeObservableValue(this);
}
abstract [Symbol.iterator](): Iterator<[K, V]>;
abstract get size(): number;
@ -94,4 +99,4 @@ export type Updater<V, MappedV> = (params: any, mappedValue?: MappedV, value?: V
export type Comparator<V> = (a: V, b: V) => number;
export type Filter<K, V> = (v: V, k: K) => boolean;
export type Filter<K, V> = (v: V, k: K) => boolean;

6
src/observable/map/ObservableMap.ts
View File

@ -86,15 +86,15 @@ export class ObservableMap<K, V> extends BaseObservableMap<K, V> {
return this._values.size;
}
[Symbol.iterator](): Iterator<[K, V]> {
[Symbol.iterator](): IterableIterator<[K, V]> {
return this._values.entries();
}
values(): Iterator<V> {
values(): IterableIterator<V> {
return this._values.values();
}
keys(): Iterator<K> {
keys(): IterableIterator<K> {
return this._values.keys();
}
}

53
src/observable/map/ObservableValueMap.ts Normal file
View File

@ -0,0 +1,53 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableMap} from "./BaseObservableMap";
import {BaseObservableValue} from "../value/BaseObservableValue";
import {SubscriptionHandle} from "../BaseObservable";
export class ObservableValueMap<K, V> extends BaseObservableMap<K, V> {
private subscription?: SubscriptionHandle;
constructor(private readonly key: K, private readonly observableValue: BaseObservableValue<V>) {
super();
}
onSubscribeFirst() {
this.subscription = this.observableValue.subscribe(value => {
this.emitUpdate(this.key, value, undefined);
});
super.onSubscribeFirst();
}
onUnsubscribeLast() {
this.subscription!();
super.onUnsubscribeLast();
}
*[Symbol.iterator](): Iterator<[K, V]> {
yield [this.key, this.observableValue.get()];
}
get size(): number {
return 1;
}
get(key: K): V | undefined {
if (key == this.key) {
return this.observableValue.get();
}
}
}

3
src/observable/map/index.ts
View File

@ -14,4 +14,5 @@ export {FilteredMap} from './FilteredMap';
export {JoinedMap} from './JoinedMap';
export {LogMap} from './LogMap';
export {MappedMap} from './MappedMap';
export {ObservableMap} from './ObservableMap';
export {ObservableMap} from './ObservableMap';
export {ObservableValueMap} from './ObservableValueMap';

87
src/observable/value/BaseObservableValue.ts Normal file
View File

@ -0,0 +1,87 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AbortError} from "../../utils/error";
import {BaseObservable} from "../BaseObservable";
import type {SubscriptionHandle} from "../BaseObservable";
import {FlatMapObservableValue} from "./index";
// like an EventEmitter, but doesn't have an event type
export abstract class BaseObservableValue<T> extends BaseObservable<(value: T) => void> {
emit(argument: T): void {
for (const h of this._handlers) {
h(argument);
}
}
abstract get(): T;
waitFor(predicate: (value: T) => boolean): IWaitHandle<T> {
if (predicate(this.get())) {
return new ResolvedWaitForHandle(Promise.resolve(this.get()));
} else {
return new WaitForHandle(this, predicate);
}
}
flatMap<C>(mapper: (value: T) => (BaseObservableValue<C> | undefined)): BaseObservableValue<C | undefined> {
return new FlatMapObservableValue<T, C>(this, mapper);
}
}
interface IWaitHandle<T> {
promise: Promise<T>;
dispose(): void;
}
class WaitForHandle<T> implements IWaitHandle<T> {
private _promise: Promise<T>
private _reject: ((reason?: any) => void) | null;
private _subscription: (() => void) | null;
constructor(observable: BaseObservableValue<T>, predicate: (value: T) => boolean) {
this._promise = new Promise((resolve, reject) => {
this._reject = reject;
this._subscription = observable.subscribe(v => {
if (predicate(v)) {
this._reject = null;
resolve(v);
this.dispose();
}
});
});
}
get promise(): Promise<T> {
return this._promise;
}
dispose(): void {
if (this._subscription) {
this._subscription();
this._subscription = null;
}
if (this._reject) {
this._reject(new AbortError());
this._reject = null;
}
}
}
class ResolvedWaitForHandle<T> implements IWaitHandle<T> {
constructor(public promise: Promise<T>) {}
dispose(): void {}
}

45
src/observable/value/EventObservableValue.ts Normal file
View File

@ -0,0 +1,45 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./index";
import {EventEmitter} from "../../utils/EventEmitter";
export class EventObservableValue<T, V extends EventEmitter<T>> extends BaseObservableValue<V> {
private eventSubscription: () => void;
constructor(
private readonly value: V,
private readonly eventName: keyof T
) {
super();
}
onSubscribeFirst(): void {
this.eventSubscription = this.value.disposableOn(this.eventName, () => {
this.emit(this.value);
});
super.onSubscribeFirst();
}
onUnsubscribeLast(): void {
this.eventSubscription!();
super.onUnsubscribeLast();
}
get(): V {
return this.value;
}
}

109
src/observable/value/FlatMapObservableValue.ts Normal file
View File

@ -0,0 +1,109 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./index";
import type {SubscriptionHandle} from "../BaseObservable";
export class FlatMapObservableValue<P, C> extends BaseObservableValue<C | undefined> {
private sourceSubscription?: SubscriptionHandle;
private targetSubscription?: SubscriptionHandle;
constructor(
private readonly source: BaseObservableValue<P>,
private readonly mapper: (value: P) => (BaseObservableValue<C> | undefined)
) {
super();
}
onUnsubscribeLast(): void {
super.onUnsubscribeLast();
this.sourceSubscription = this.sourceSubscription!();
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
onSubscribeFirst(): void {
super.onSubscribeFirst();
this.sourceSubscription = this.source.subscribe(() => {
this.updateTargetSubscription();
this.emit(this.get());
});
this.updateTargetSubscription();
}
private updateTargetSubscription(): void {
const sourceValue = this.source.get();
if (sourceValue) {
const target = this.mapper(sourceValue);
if (target) {
if (!this.targetSubscription) {
this.targetSubscription = target.subscribe(() => this.emit(this.get()));
}
return;
}
}
// if no sourceValue or target
if (this.targetSubscription) {
this.targetSubscription = this.targetSubscription();
}
}
get(): C | undefined {
const sourceValue = this.source.get();
if (!sourceValue) {
return undefined;
}
const mapped = this.mapper(sourceValue);
return mapped?.get();
}
}
import {ObservableValue} from "./ObservableValue";
export function tests() {
return {
"flatMap.get": (assert): void => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const countProxy = a.flatMap(a => a!.count);
assert.strictEqual(countProxy.get(), undefined);
const count = new ObservableValue<number>(0);
a.set({count});
assert.strictEqual(countProxy.get(), 0);
},
"flatMap update from source": (assert): void => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
a.flatMap(a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
assert.deepEqual(updates, [0]);
},
"flatMap update from target": (assert): void => {
const a = new ObservableValue<undefined | {count: ObservableValue<number>}>(undefined);
const updates: (number | undefined)[] = [];
a.flatMap(a => a!.count).subscribe(count => {
updates.push(count);
});
const count = new ObservableValue<number>(0);
a.set({count});
count.set(5);
assert.deepEqual(updates, [0, 5]);
}
};
}

71
src/observable/value/MapSizeObservableValue.ts Normal file
View File

@ -0,0 +1,71 @@
/*
Copyright 2022 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./index";
import {BaseObservableMap} from "../map/index";
import type {SubscriptionHandle} from "../BaseObservable";
export class MapSizeObservableValue<K, V> extends BaseObservableValue<number> {
private subscription?: SubscriptionHandle;
constructor(private readonly map: BaseObservableMap<K, V>)
{
super();
}
onSubscribeFirst(): void {
this.subscription = this.map.subscribe({
onAdd: (key: K, value: V) => {
this.emit(this.get());
},
onRemove: (key: K, value: V) => {
this.emit(this.get());
},
onUpdate: (key: K, value: V) => {},
onReset: () => {
this.emit(this.get());
},
});
}
onUnsubscribeLast(): void {
this.subscription = this.subscription?.();
}
get(): number {
return this.map.size;
}
}
import {ObservableMap} from "../map/index";
export function tests() {
return {
"emits update on add and remove": assert => {
const map = new ObservableMap<string, number>();
const size = new MapSizeObservableValue(map);
const updates: number[] = [];
size.subscribe(size => {
updates.push(size);
});
map.add("hello", 1);
map.add("world", 2);
map.remove("world");
map.remove("hello");
assert.deepEqual(updates, [1, 2, 1, 0]);
}
};
}

82
src/observable/value/ObservableValue.ts Normal file
View File

@ -0,0 +1,82 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {AbortError} from "../../utils/error";
import {BaseObservableValue} from "./index";
export class ObservableValue<T> extends BaseObservableValue<T> {
private _value: T;
constructor(initialValue: T) {
super();
this._value = initialValue;
}
get(): T {
return this._value;
}
set(value: T): void {
if (value !== this._value) {
this._value = value;
this.emit(this._value);
}
}
}
export function tests() {
return {
"set emits an update": (assert): void => {
const a = new ObservableValue<number>(0);
let fired = false;
const subscription = a.subscribe(v => {
fired = true;
assert.strictEqual(v, 5);
});
a.set(5);
assert(fired);
subscription();
},
"set doesn't emit if value hasn't changed": (assert): void => {
const a = new ObservableValue(5);
let fired = false;
const subscription = a.subscribe(() => {
fired = true;
});
a.set(5);
a.set(5);
assert(!fired);
subscription();
},
"waitFor promise resolves on matching update": async (assert): Promise<void> => {
const a = new ObservableValue(5);
const handle = a.waitFor(v => v === 6);
await Promise.resolve().then(() => {
a.set(6);
});
await handle.promise;
assert.strictEqual(a.get(), 6);
},
"waitFor promise rejects when disposed": async (assert): Promise<void> => {
const a = new ObservableValue<number>(0);
const handle = a.waitFor(() => false);
await Promise.resolve().then(() => {
handle.dispose();
});
await assert.rejects(handle.promise, AbortError);
},
};
}

89
src/observable/value/PickMapObservableValue.ts Normal file
View File

@ -0,0 +1,89 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {BaseObservableValue} from "./index";
import {BaseObservableMap, IMapObserver} from "../map/BaseObservableMap";
import {SubscriptionHandle} from "../BaseObservable";
function pickLowestKey<K>(currentKey: K, newKey: K): boolean {
return newKey < currentKey;
}
export class PickMapObservableValue<K, V> extends BaseObservableValue<V | undefined> implements IMapObserver<K, V>{
private key?: K;
private mapSubscription?: SubscriptionHandle;
constructor(
private readonly map: BaseObservableMap<K, V>,
private readonly pickKey: (currentKey: K, newKey: K) => boolean = pickLowestKey
) {
super();
}
private updateKey(newKey: K): boolean {
if (this.key === undefined || this.pickKey(this.key, newKey)) {
this.key = newKey;
return true;
}
return false;
}
onReset(): void {
this.key = undefined;
this.emit(this.get());
}
onAdd(key: K, value:V): void {
if (this.updateKey(key)) {
this.emit(this.get());
}
}
onUpdate(key: K, value: V, params: any): void {
this.emit(this.get());
}
onRemove(key: K, value: V): void {
if (key === this.key) {
this.key = undefined;
// try to see if there is another key that fullfills pickKey
for (const [key] of this.map) {
this.updateKey(key);
}
this.emit(this.get());
}
}
onSubscribeFirst(): void {
this.mapSubscription = this.map.subscribe(this);
for (const [key] of this.map) {
this.updateKey(key);
}
}
onUnsubscribeLast(): void {
this.mapSubscription!();
this.key = undefined;
}
get(): V | undefined {
if (this.key !== undefined) {
return this.map.get(this.key);
}
return undefined;
}
}

33
src/observable/value/RetainedObservableValue.ts Normal file
View File

@ -0,0 +1,33 @@
/*
Copyright 2020 Bruno Windels <bruno@windels.cloud>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import {ObservableValue} from "./index";
export class RetainedObservableValue<T> extends ObservableValue<T> {
constructor(initialValue: T, private freeCallback: () => void, private startCallback: () => void = () => {}) {
super(initialValue);
}
onSubscribeFirst(): void {
this.startCallback();
}
onUnsubscribeLast(): void {
super.onUnsubscribeLast();
this.freeCallback();
}
}

Some files were not shown because too many files have changed in this diff Show More