Matrix Live 🎙

It's demos this week!

• Riot X video calls (valere)
• Room list (TravisR)
• Notifications - (Michael (t3chguy) & richvdh)
• E2E Onboarding (Dave & Nad)
• Modular SAML support (ChristianP)

Dept of Status of Matrix 🌡

Merch Returns!

Yes it's back! For the first time in months we are shipping branded IRL-items featuring Matrix branding. Head to The Shop to stock-up.

Dept of Spec 📜

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• MSC2631: Add default_payload to PusherData

MSCs in Final Comment Period:

• MSC2604: Accept device information for the login fallback endpoint (merge)

New MSCs:

• [WIP] MSC2654: Unread counts

Spec Core Team

In terms of Spec Core Team MSC focus for this week, unfortunately our three MSCs from last week ( MSC2366 (verification flows), MSC2403 (knocking), and MSC2630 (SAS security)) are still pre-FCP. Most of the team has been quite busy with implementation for the past few weeks. Instead of advertising those 3 MSCs again, we're going to switch the focus to "implementation" for a bit until we're freed up again.

So the focus for this week is: MSC implementation work. However, this should not discourage any MSC authors from responding to MSC feedback in the meantime 🙂

Dept of GSoC 🎓

Ruma GSoC

devinr528 reported:

This week in the ruma/matrix Google Summer of Code project, ruma-events was made ready for use! After adding stripped and sync event generation to the event_enum! macro there were only a few small tweaks needed to try it out in some dependent rust crates. I spent a few days converting matrix-rust-sdk to use the ruma-monorepo. Since ruma is used on both client and server-side, I also opened a PR to update Conduit, a homeserver implementation written in Rust. To test that everything worked together, I updated rumatui, my command-line client written in rust.

Then I could test that Conduit sent, and matrix-rust-sdk received the new ruma events successfully. While updating, I felt the pain of not having accessor methods for the Any*Event enums to get at the event fields held within. I have opened a pull request to add the generation of these methods to the event_enums! macro. Hopefully, the Conduit and matrix-sdk PR's will be merged and the ruma monorepo can be tested in the wild!

matrix-ircd GSOC

karlik told us:

Moved matrix protocol, irc protocol, and bridge module to futures 0.3. Converted some utility functions to new futures, and updated the http implementation to use the standardized Hyper library instead of using a custom http implementation.

go-neb

nikofil said:

• Created PR for enabling e2ee across all services! This required a few changes to how the bot client is initialised, as well as changes to all services to use the new functionality instead of directly sending messages to a room. https://github.com/matrix-org/go-neb/pull/324

• Added code from another of tulir's projects to Mautrix to allow storing the crypto material (olm / megolm sessions, accounts etc.) in a SQL database, adding a second way besides using Gob storage. https://github.com/tulir/mautrix-go/pull/10

• Future plans are to work on the library itself to add any features that might be missing.

HTML embeddable chat rooms

arnav-t told us:

This week I've worked on the following features for the project "HTML embeddable chat rooms" under GSoC '20:

Added support for signing-in

Via sign-in popup

The user can enter their credentials into the modal dialogue box.

Via postMessage interface

iframe.contentWindow.postMessage({
    cmd: 'login',

    args: {
        user: 'username',

        passwd: 'password'
    }

}, origin);

Arguments:

args (object):

• username (string) - Username

• passwd (string) - Password

Response:

{

    status: 'success', 
    message: 'Attempting sign in...'

}

Added support for default avatars

Using the same design as Riot. However, I am just using flexbox and text instead of drawing on an HTML5 canvas. The HSL color is generated by hashing the user ID.

Quoting and replying

Added on-hover buttons and reply popup above message composer similar to Riot web. Quoting prepends the message (quoted) to the message composer similar to Riot web and replying would also work similar to Riot web. It will be fully functional once markdown parsing is added.

Guest access

The client now supports guest access. If no access token or user ID is provided in the configuration file, the client attempts to register a guest account on the home server. Room contents are viewed using peekInRoom.

If the guest attempts to send a message, only then joinRoom is called to avoid spam (each page load would lead to a guest joining the room).

Nheko

Chethan told us:

SAS Verification is completed and works !

Supports both Emoji and Decimal Verification Only uses curve25519-hkdf-sha256 as key_agreement protocol

Brand new UserProfile written entirely in qml. https://github.com/Nheko-Reborn/nheko/pull/203

Mtxclient

Updated with needed APIs for SAS verification.

https://github.com/Nheko-Reborn/mtxclient/pull/32

Opsdroid

tyagdit reported:

Port to Matrix-nio

• The matrix connector is now using matrix-nio!

• Bots can send and receive texts, images, files as before

• No change in configuration required, just update opsdroid and run as before

• Encryption is not yet implemented

• You can check out the PR here

Enabling E2EE

• Sending and receiving encrypted texts, edits, replies works just as normal messages

• Images and files are being worked on

• User shouldn't have to change much about their configuration to use encryption

• Some testing is due

• Check out the progress here

Dept of Servers 🏢

Dendrite

Dendrite is a next-generation homeserver written in Go

kegan reported:

This week has been mainly about testing to ensure that all the progress over the past few months is kept up-to-date and correct. We've also added a few features in order to get certain sytests working. To that end:

• Invites can now be declined over federation and they will be reflected in /sync responses.

• Errors encountered when joining a room over federation are now sent back to the client.

• Errors encountered when accepting an invite over federation are now sent back to the client.

• Dendrite will now check server names meet the server name grammar in the specification.

• A bug which caused client-api-proxy to not actually proxy correctly has been fixed, thanks @fantashley !

• /send now abides by the limits in the specification: 50 PDUs / 100 EDUs.

• The docker-compose scripts now include appservice_api, thanks @fantashley !

• Sending invites over federation will now fall back to v1 if v2 fails with a 404.

• Dendrite now implements room.timeline.limit completely (in both in-line and stored filter formats).

• Dendrite now sets the limited flag on /sync responses correctly.

In addition, we now have support for collecting code coverage output from SyTest. This indicates we are testing roughly 70% of the Dendrite codebase. The remaining 30% are hard to reach via intergration tests (e.g database failures, communication problems between internal APIs).

Spec compliance:

• Client-Server APIs: 45%, up from 40% last week

• Server-Server APIs: 50%, up from 38% last week

In total, we've made an additional 45 sytests pass this week.

Mascarene

Nico reported:

Mascarene v0.2.0 has been released.

This version implements a minimal set of client API endpoints to work with Riot and allow user registration,

room creation, invitation and messages sending on the same running instance (no federation support).

Docker image are also available. See this documentation

for rough installation instructions.

Join us at #mascarene:beerfactory.org

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

This week I finished the cross-signing PR:

To try it out yourself, open Riot (preferably https://riot.im/develop) and join the <https://conduit.koesters.xyz:14004> homeserver.

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil told us:

This week we’ve been working on further improvements to event persistence and ironed out a nasty bug where an unusually long state resolution could block the reactor overall and impact send times. We seem to have got to the bottom of this and m.org has improved a lot as of today.

More generally we’ve been trying to characterise matrix.org performance so that we can continue to improve over the coming months.

We are going to focus on:-

• Client send event

• Outbound Federation Latency

• Inbound Federation Latency

• Room joins

As well as tracking the CS API generally.

We are using apdex with a satisfied limit of 250 ms and a tolerating limit of 1000 ms. By the end of the Summer we will aim to hit an apdex score of 0.9 for each area.

For instance here is Federation Send Event Apdex graphed overtime. You can see that we are averaging about ~0.8 currently, so plenty to keeping us busy!

Next up will be to work on Outbound Federation Latency

Aside from that we’ve been working hard on the upcoming Notifications improvements. Mapping the push rules to the demands of the UI has been challenging and we’ve been through several iterations. If you’d like to learn more take a look at Michael and Rich’s explanation in this week’s Matrix Live. Rich’s presentation has Sheltie pictures #justsayin’

Dept of Bridges 🌉

It's a Tulir kind of week.

mautrix-whatsapp

Tulir told us:

The WhatsApp bridge got a bunch of improvements this week, such as:

• Matrix -> WhatsApp gif bridging

• Bridge status notices if your phone battery is low

• A command to get group invite links and another command to join groups using invite links

• Matrix <-> WhatsApp bridging of most membership changes

mautrix-facebook

Tulir reported:

The Messenger bridge mostly got bugfixes, but also Matrix->Facebook bridging of all media and location messages

mautrix-telegram

Tulir offered:

The Telegram bridge now supports logging in by scanning a QR code, although it requires using the master branch of Telethon instead of a release. I also fixed bridging captions in file messages, so they're now bridged as separate messages like with images

Half-Shot bridges roundup

Half-Shot offered:

Hey all, I've got several releases to talk about this week!

First, matrix-appservice-slack has been updated to 1.4.0 with several quality of life changes such as automatically setting the bot profile on startup, supporting logging out of slack accounts and adding a health checkpoint. There are also quite a few bugfixes so make sure you update.

matrix-appservice-irc was also bumped to 0.18.0 with the headline features being Node 14.x support.

We've also shipped 0.19.0-rc1 which has stopped support for Node 10.x. Why? Because we're adding worker support to the bridge! This release starts to make use of the new(ish) worker_threads feature so that we can dish out processing to seperate threads (running their own UV event loops, for node enthusiasts). The first thing to be workerized is metrics, so that metrics may still be reported should the bridge become saturated, but we plan to split out more work as things progress.

Finally matrix-appservice-bridge got a few fixes to support our new worker land, as well as being updated to support matrix-js-sdk 6.0.0. You can checkout the changes for 1.13.1 here.

Dept of Clients 📱

Fluffychat

sorunome offered:

Fluffychat Version 0.15.0 is released, and should be available in the Play Store, on F-Droid and in IOS Testflight soon! This makes Fluffychat the first non-Riot matrix client that supports Cross-Signing.

Features:

• New room list app bar design

• Chat app bar transparent

• Implement web file picker

• Minor design and UX improvements

• Implement Cross Signing

• Restore keys from online key backup

• Added translations: Czech, Spanish, Slovakian

Changes:

• Show presences of users sharing a direct chat

• Big refactoring

Fixes:

• Various fixes, including e2ee fixes and olm session recovery

Pattle: version 0.18.0

Wilko said:

A new version has been released and will soon become available on Google Play, TestFlight and F-Droid!

Changes

• You can now play videos!

• The main public address (room alias) of chats are now shown in chat's details

• The time of sending is now always show on the right side for images

• Notifications are grouped nicer, there's now one notification for Pattle, grouped by chat

• Fix sending an image crashing the app

• Fix urls not being accounted for in time placement (#132)

• Fix chat input not scrolling

• Fix multiple issues regarding showing notifications

• F-Droid: All left-over proprietary code is now removed, thanks to Bubu for notifying me!

Get Pattle

• Android

  • F-Droid
  • Google Play

• iOS (TestFlight)

Get involved

• Follow development: #app:pattle.im

• Issues or suggestions: Post them here!

• Donate: If you want to support Pattle, you can donate!

Mirage

miruka* reported:

0.5.2 is now out:

Added

• Sessions/device list: you can now inspect, rename and manually verify

  your devices from the account settings page. The interface is still work in progress, keyboard navigation and signing

  out sessions will be added in a next version.

• Re-add client-side unread/highlight room indicators.

  If your account has push notifications disabled, which precise cross-client counters depend on, the local indicators will be used as fallback.

• Support the MIRAGE_CACHE_DIR environment variable to override where

  files and thumbnails are downloaded

• A bunch of theme additions, check the full log

Changed

• Overhauled account settings to match the design of other tabbed pages.

  The horizontal layout design has been removed due to complicated code and being impossible to extend without breaking it.

• The display name field in account settings is now colored,

  preview your new display name's color as you type

• For rooms without image avatars set, the room settings's avatar color now

  responds to the name field as you type

• Overhauled scrollbars:

  • Now match the Mirage theme and much better visibility

  • No more right margin for the timeline's bar

  • Minimum height to prevent the bar from becoming impossible to grab

• Use brighter text for room names of rooms that have unread messages

• Buttons, tabs, text fields and areas now have animated bottom borders

  to represent keyboard focus instead of being highlighted like when hovered

• Text fields and areas can now have rounded corners, following the theme

• Tabbed pages (Sign In, Add Chat, etc) can now be swiped left and right

• Popups can now be scrolled when their content is bigger than the

  window's height

• Replace most generic checkmark icons for apply buttons in popups

• Pressing escape in forms will consistently trigger corresponding

  cancel buttons

Fixed

• Various fixes, see full changelog

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) offered:

• Most of our time this week was spent trying out device verification with Chethan. It's fun and I could finally file bugs in Nheko against someone else!

• We fixed some issues that should hopefully make text in the timeline less blurry again.

• Fixed a bug which cause some clients like fluffychat to break in E2EE rooms (we accidentaly sent a null relation when not replying...)

• Fixed an issue, where Nheko didn't verify the format of html formatted messages correctly, causing it to render messages in a way which wasn't compliant to the spec.

• Thanks to the work of a Pirate and his friends, current versions of Nheko should be available in backports for Debian Buster once again!

Riot Chat for Nextcloud

Gary Kim announced:

Riot Chat for Nextcloud 0.5.0 The new version updated the Riot.im version to 1.6.6 and added the ability for admins to set their own custom config for Riot rather then using the settings interface in addition to a few bug fixes.

Riot Chat for Nextcloud allows individuals and organizations with a Nextcloud instance to easily set up and configure their own Riot instance with just a few clicks on a web interface without the need to write a config file.

Join the development Matrix room at #riotchatfornextcloud-general:garykim.dev. Check out the source code here.

Riot-iOS

Manu told us:

This week, we completed UX for cross-signing and secure backup. We made associated settings but we still need to polish them.

Sygnal and the new push extension have been updated to match Apple requirement and our privacy concerns. Events content are no more sent anymore. We started to implement the new room notifications settings UI.

RiotX

benoit told us:

This week we were mainly working on cross-signing, room settings, VoIP stabilization and performance improvement.

Riot-Android

benoit reported:

We will publish at the beginning of next week a beta version of the migration to RiotX codebase on the beta channel of the PlayStore, to be able to ensure the migration works fine, before release it to production.

Dept of SDKs and Frameworks 🧰

Matrix Dart SDK: v0.33.0

Wilko told us:

• Support custom events using RawRoomEvent and RawStateEvent

  Check the README for details.

• Add ability to register using Homeserver.register

• Add VideoMessageEvent

• Add Room.canonicalAlias and Room.alternativeAliases

• Add format field to TextMessage (thank you Cyril!)

• Remove equatable dependency

Get it on pub!

Net::Matrix::Webhook

joepie91 discovered this project, presented yesterday at Conference in the Cloud:

Net::Matrix::Webhook implements a webhook, so you can easily post messages to your matrix chat rooms via HTTP requests. It uses IO::Async to start a web server and connect as a client to matrix. It will then forward your messages.

matrix_scrape_emoji_translations script project

sorunome announced:

Soru wrote a quick program that scrapes riot-web, riot-x and riot-ios for translations of the emoji names for emoji verification and combines them all into an easily-readable json file, so that other client developers can use it. Since all three riot versions have a different set of translations, it might also be helpful for them. You can find the source code along with the outputted json files here.

Dept of Bots 🤖

"Scandinavia and the World" comic bot

Tulir offered:

To add to my existing xkcd and CommitStrip maubot plugins, there's now a similar plugin for Scandinavia and the World comics: https://github.com/maubot/satwcomic

Like the other bots, you can self-host it, use my instance (@satw:maunium.net) in your own room or simply join #satwcomic:maunium.net to automatically get the latest comics in Matrix.

matrix-webhooker

kinta announced:

A bot that will allow to room administrators and moderators to generate room custom commands (in a similar way how telegram commands are thought).

When they are invoked it will post the message event object data and a predefined context object along with a token to a custom url. More information in: https://gitlab.com/communia/matrix-webhooker

Matrix webhook

kinta told us:

https://www.drupal.org/project/matrix_webhook

Drupal module to receive links shared from a matrix room. Get links provided by a matrix bot in a room. With this module one can get media from a matrix own bot to any drupal site. Once installed and bot is created through matrix-webhooker bot. A blog entry about it (in spanish) in https://planet.communia.org/content/enlaces-desde-matrix

Dept of Interesting Projects 🛰

Pollvis - new poll visualiser project

We have a couple of conferences coming up who are planning to use Matrix as part of their offering, much as we do with Open Tech Will Save Us. To help add some more features, I created an MVP "poll visualiser", which watches a room and works in tandem with the poll-bot from Brendan Abolivier . This project is still at the beginning, but might be interesting to some! Find the code at https://github.com/benparsons/pollvis.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Matrix Live 🎙

Dept of Status of Matrix 🌡

Matrix Merch Store is back

Are you emerging from lockdown? Dazed and wondering what to wear? Need some apparel which says "I'm cool and I have great taste in decentralised communications protocols"? Then you should visit The Matrix Merch Store!

T-Shirts, Hoodies, Zipped Hoodies and Stickers available now!

Matrix Decomposition: Analysis of an Access Control Approach on Transaction-based DAGs without Finality

Florian told us:

We published another scientific paper on Matrix, this time it is called Matrix Decomposition: Analysis of an Access Control Approach on Transaction-based DAGs without Finality! Its main topic is the question of how access control can work based on a structure that only provides a partial order on events and no consensus on the current state, i.e. the Matrix Event Graph. We found some concrete, non-critical security issues related to both incorrect specification as well as divergent homeserver behaviour. While the last remaining mitigations found their way into Synapse 1.14.0, we provide ideas on structural solutions to avoid both problem classes in the future using formal verification.

The paper was published at the ACM Symposium on Access Control Models and Technologies(SACMAT) last week. My talk went well (🎉), it was recording and is yet to be published by the conference.

The paper was previously mentioned on the Matrix blog.

Dept of Spec 📜

anoa reported:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• MSC2399: Reporting that decryption keys are withheld
• MSC1703: encrypting recovery keys for online megolm backups

MSCs in Final Comment Period:

• No MSCs are in FCP.

New MSCs:

• MSC2644: matrix.to URI syntax v2
• MSC2638: Ability for clients to request homeservers to resync device lists

Spec Core Team

In terms of Spec Core Team MSC focus for this week, the team has been spread a bit thin with other work. As such, we're still on the same three MSCs: MSC2366 (verification flows), MSC2403 (knocking), and MSC2630 (SAS security). We're hoping this week will be more productive in terms of spec.

Dept of Servers 🏢

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander announced:

This week has seen a lot of work put into fixing broken tests and refactoring parts of the codebase.

• The media API has had quite a bit of attention this week and now media over federation works properly, as well as much better filename handling

• A new user API component is now responsible for user accounts, devices, access tokens and account data

• The server key API has been updated with more reliable validity checking and new tests

• Logging into Dendrite using Riot iOS is now supported thanks to some minor tweaks/fixes

• URL handling for version 3 rooms has been improved

• A bug in the format of /v1/send_join has been fixed

• The /joined_members response has now been fixed

In addition to that, we've made good progress on embedding the Dendrite-powered Yggdrasil demo into Riot iOS, which we'll hopefully be demoing in a couple of weeks!

Spec compliance has improved:

• Client-Server APIs: 40%, up from 36% last week

• Server-Server APIs: 38%, up from 35% last week

Synapse

Neil offered:

Highlights this week

• We released 1.15.1, this is a bug fix release, sorry if you were bitten.

Changelog in full

Bugfixes

• Fix a bug introduced in v1.15.0 that would crash Synapse on start when using certain password auth providers.

• Fix a bug introduced in v1.15.0 which meant that some 3PID management endpoints were not accessible on the correct URL.

Outside of that

• Implemented unread message counts (MSC 2625) - This is part of the a more general notifications improvement project in conjunction with the Riot teams. The aim being to make Notifications easier to configure. We’ve also been working on push rule definitions for the default behaviour. More on that next week.

• Have the ability to shard the media repository worker. This means we can now run an arbitrary number in parallel, thereby improving upload reliability and performance, we are running this on Matrix.org.

• Re-enabled Jaeger on matrix.org - previously we needed to switch it off due to the performance overhead, we needed a few tweaks but now it is working well and helping us to determine the cause of slow requests. It seems like the HTTP requests between the event creator and event persister workers can sometimes be slow - we are not sure of the cause yet, we are working on it but it will mean further performance improvements to message sending.

• Next up performance wise is working on sharding the federation sender to improve federation lag. We’ll also try replacing simplejson with stdlib json which seems to be much faster at dumps nowadays.

• Finally, we’ve fixed a bug that meant that quarantining media would from time to time include quarantining sticker packs

Other interesting bugs

We fixed a spec compliance bug in fetching remote media raised by our pals in Dendrite-land. We fixed a bug causing federation .well-known requests to fail due to not including the user-agent header. We removed references to six (thanks @ilmari) and finally, fixed a bug that caused stream id to go backwards on the replication stream.

Next week we’ll continue to work on notifications and performance. You can track our progress here https://github.com/orgs/matrix-org/projects/8

Synapse adoption graphs

Chris said:

Wondering how quick this weeks synapse release was adopted by homeserver admins? Head over to https://graph.settgast.org/ and see yourself that it only took a little over 2 days for 1.15.1 to become the most used synapse version. I plot hourly distribution of synapse versions there. (Disclaimer: Only collects stats for homeservers that my homeserver sees in any of its rooms, which are around 2000 at the moment)

This is super cool! I love that this has been created, and also how well it reflects on homeserver admins! /me scurries to update to 1.15.1...

Asked to share the process, Chris sent:

Its a horribly hacky python script, but I can at least share it here: https://gist.github.com/csett86/96190592390b2c3d8c12c020ce312723

What version are you running?

Synapse on a ROCKPro64

How is Synapse performance these days? PINE64 are running their instance on their own ROCKPro64 hardware.

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo announced:

This week I worked on server-side key backups and cross-signing - the second most requested features (federation being the first one). Key backups are already working! Take a look:

Cross-signing should be working, but there is a bug where emoji verification gets stuck. I hope we can find the mistakes and finish the PR next week.

Thanks to everyone who supports me on Liberapay or (new!) Bitcoin!

Synapse Deployment 📥

Kubernetes

Ananace said:

Just pushed 1.15.1 tags for the K8s-optimized Synapse image.

Docker-matrix

Mathijs announced:

The image for synapse v1.15.1 including jemalloc and mjolnir anti-spam has been released to avhost/docker-matrix:v1.15.1.

Dept of Bridges 🌉

matrix-ircd, GSOC project

karlik offered:

For matrix-ircd google summer of code:

• Began moving the codebase to the more modern futures 0.3 (previously futures 0.1)

• Introduced more modern async / await syntax for improved code clarity

• Wrote some additional unit tests, doubling the overall test coverage to 32%

• Updating the 2016 code to use rust 2018 idioms PR # 64

• Swapped repository to use stable rust (previously nightly) PR #65

mautrix bridges

Tulir told us:

To make the bridge info state events a bit more useful, all my bridges now include the room name and avatar URL in them. The events are also updated whenever the name or avatar changes, and there's a new option to re-send the event to all existing portals so old portals would get it too.

Dept of Clients 📱

RiotX: Version 0.22.0

benoit said:

RiotX: Version 0.22.0 has been released on Tuesday, it includes integration manager support, sending stickers, modifying power levels, and lots of other things! See https://github.com/vector-im/riotX-android/releases/tag/v0.22.0 for a complete list of the changes.

This week we've been working on audio and video calls (The PR is in review). We are also trying to improve the performance of the application. We are adding room settings, and we have started to work on the migration Riot-Android -> RiotX.

PS check out Valere trying the new features:

Arch Linux (AUR) package for Revolution - a Riot fork by ponies.im

Sophie still alive said:

I made an Arch Linux (AUR) package for Revolution - a Riot fork by ponies.im:

• https://aur.archlinux.org/packages/revolution-desktop-git/

• https://aur.archlinux.org/packages/revolution-web-git/

• https://gitlab.eastcoast.hosting/ponies

Daydream

MTRNord told us:

Daydream is getting a new Design and the first page is ready at https://app.daydream.im/

Additional changes are:

• Removed client side js (except what's needed to load wasm stuff)

• Replaced loading animation with css only version

• Replace icons with css only version

• Add e2e tests

• Add Basic Notifications

Currently on top of the TODO list of Daydream is:

• Adding a wasm compatible (indexed db based) state store

• Notifications

Rumatui is basically usable

WIP Command line Matrix client using matrix-rust-sdk

devinr528 told us:

After some slow progress, rumatui, the rust command line chat client, is usable by anyone. You no longer need an account or have previously joined rooms. Registration and room search have been implemented! Using left/right arrow keys from the login screen will toggle to registration. Once registered hitting the left arrow will bring up the room search window. After typing your search term hit Enter and select the room to join by pressing Ctrl-d.

For the brave who would like to give it a try cargo install --git <https://github.com/DevinR528/rumatui> --branch room-search.

Added since last update

• Room search is now available

  • Public rooms can be joined from the room search window

• A user can register from the new register window

  • This features complete User Interactive Authentication by opening a web browser

• Message edits are shown

  • When markdown is part of the message they are properly formatted

• Reactions display under the respective message

• Redaction events are handled for reactions (emoji) and messages

"Basically usable" is a standard we can all aspire to for our projects and endeavours.

Riot-iOS

Manu told us:

This week, we worked on the new UX for cross-signing and secure backup. In parallel, we continued to improve the new push implementation. We are very closed to complete the two features.

gomuks

gomuks is a terminal based Matrix client written in Go. Source on GitHub

Tulir announced:

I added a command to add and remove local aliases for rooms. Changes to alt_aliases are also now rendered properly.

Riot Web

Ryan offered:

This week we had an off-cycle 1.6.5 release to fix registration on some homeservers when email validation is required. After that, we've made a 1.6.6 RC including:

• Added support for new emoji in Unicode 13

• Emoji picker now supports searching by emoji characters (to find related ones) as well as text

• Added a homeserver admin .well-known option to control default E2EE behaviour

• Many more bug fixes and smaller improvements

Riot Web Themes

jo offered:

made a couple riot themes! Dracula and Dracula Flat

https://raw.githubusercontent.com/aaronraimist/riot-web-themes/master/Dracula/Non-flat/Dracula.json https://raw.githubusercontent.com/aaronraimist/riot-web-themes/master/Dracula/Flat/DraculaFlat.json

Find images here and here

Go to https://github.com/aaronraimist/riot-web-themes to find out how to use these.

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) offered:

• We released 0.7.2, which was a bit more messy than we would have liked...

• As with every release we are currently working through the new bug reports. :D

• I started working on SSSS and online key backup support. For this I wrote my own base64 and base58 encoders. As a result of this we dropped libsodium as a dependency, which should make Nheko a bit easier to package or build. It's also fun to write base conversions, although 58 is a horrible base!

From the changelog:

• Reactions
  • React to a message with an emoji! 🎉
  • Reactions are shown below a message in a small bubble with a counter.
  • By clicking on that, others can add to the reaction count.
  • It may help you celebrating a new Nheko Release or react with a 👎 to a failed build to express your frustration.
  • This uses a new emoji picker. The picker will be improved in the near future (better scrolling, sections, favorites, recently used or similar) and then probably replace the current picker.
• Support for tagging rooms [tag]
  • Assign custom tags to rooms from the context menu in the room list.
  • This allows filtering rooms via the group list. This puts you in a focus mode showing only the selected tags.
  • You can assign multiple tags to group rooms however you like.
• SSO Login
  • With this you can now login on servers, that only provide SSO.
  • Just enter any mxid on the server. Nheko will figure out that you need to use SSO and redirect your browser to the login page.
  • Complete the login in your browser and Nheko should automatically log you in.
• Presence
  • Shows online status of the people you are talking to.
  • You can define a custom status message to tell others what you are currently up to.
  • The status message appears next to the usernames in the timeline.
  • Your server needs to have presence enabled for this to work.

Wowow that's a lot this time!

Dept of SDKs and Frameworks 🧰

quotient

kitsune said:

after a couple of setbacks that didn't let us to make the releases earlier, the Quotient project has finally made two new releases:

• GTAD (the piece of code magically producing readable C++ code from a Swagger API description) has achieved version 0.7 (https://github.com/KitsuneRal/gtad/releases/tag/v0.7.0) adding a few tricks in order to make...

• ...libQuotient 0.6 beta2 (https://github.com/quotient-im/libQuotient/releases/tag/0.6-beta2) rely entirely on the upstream matrix-doc specification, rather than a soft fork closely following the main sources! From now on it's "upstream first", in a sense that the original matrix-doc will be used to build Quotient codebase. Let's see how often it breaks ;) In other news from the last few weeks:

• some housekeeping and deprecation work in the API has been done in preparation to getting User Interactive Authentication along the next (post-0.6) release cycle of libQuotient.

• also thanks to the updated code generator, the CS API code has been optimised, consolidating more code in the header files and making data deserialisation lazy; this helped significantly reduce compilation times, and runtime performance also improved a bit.

• the number of configurations tested by CI has been extended, allowing to chase down a few more bugs that managed to go under the radar before.

• when used with new enough Qt, CBOR is used to cache data locally - entirely transparently for clients.

Expect more news in the coming weeks, including continued work on matrix: URI proposal and its implementation in Quotient.

ruma-events' Google Summer of Code project

devinr528 sent us this late entry!

This week in ruma-events' Google Summer of Code project, after trying out the new Any*Event enums matrix-rust-sdk, we found a few big flaws. There was no easy way to go from StateEvent<AnyStateEventContent> to StateEvent<SpecificEventContent>, the other issue was one could create a StateEvent with differing content and prev_content fields using the AnyStateEventContent enum. The 0.22 ruma-events will be similar to the existing API; each event type has a corresponding event enum variant.

pub enum AnyStateEvent {
    RoomMember(StateEvent<MemberEventContent>),
    RoomAliases(StateEvent<RoomAliases>),
    // ...
}

There were a few minor fixes during the week also. Unknown field deserialization is fixed, allowing deserialization of a JSON blob that has extra fields which are ignored. Custom events are now present in the Any*EventContent enums, although now they have to be moved up to be included in Any*Event enums. Benchmarks for deserialization have been added and used to increase performance.

🧙 Polyjuice Client

Polyjuice Client is a Matrix library for Elixir

uhoreg announced:

Polyjuice Client, a Matrix library for Elixir, had a new release. This release fixes syncs, which apparently were completely broken in the last release (whoops). The wizarding community also welcomes a new contributor, Pierre de Lacroix.

Dept of Ops 🛠

Gitea

s7evink reported:

Gitea 1.12.0 was finally released. It's now possible to send notifications (commits, pull requests, etc.) directly to Matrix using webhooks.

Dept of Internet of Things 💡

Mozilla Thing Matrix Adapter

Christian offered:

I heard that you people like IoT?! With the Mozilla Thing Matrix Adapter your Raspberry Pi sends you messages about your home. Want a log of when the front door has been opened? Need a low battery alert for your IoT devices? With the Mozilla IoT gateway and this adapter, you can send these events to a Matrix room of your choice!

https://gitlab.com/webthings/matrix-adapter

Dept of Bots 🤖

Alt alias maubot plugin

Tulir announced:

Ever since m.room.aliases was replaced with alt_aliases, there hasn't really been any way to find aliases on other servers, since most room admins don't bother finding and publishing alternate aliases. To help solve that problem, I made a maubot plugin to let users publish alternate aliases in rooms: https://github.com/maubot/altalias

By default it lets users publish aliases with the same localpart as any already published alias. If that behavior is sufficient, you can simply invite it to a room and give it permissions to send m.room.canonical_alias events. Alternatively, it can be configured to allow aliases that match specific regexes.

Once it's set up, users can create a local alias and use !altalias publish <their alias> to publish it. The bot will make sure the alias points to the right room, check that the localpart is allowed and then add it to alt_aliases.

I have it running at @alias:maunium.net and I've added it to all my rooms already. Other rooms are also free to use that instance.

go-neb on NixOS

maralorn offered:

As of https://github.com/NixOS/nixpkgs/pull/89327 a go-neb module was added to nixos and will soon be available on nixos-unstable and in the 20.09 release. You can use it to declaratively install and configure go-neb matrix bots on nixos. I use it for prometheus alertmanager alerts and it works really cool! Thx to hexa- who did most of the work on it.

Opsdroid

Cadair offered:

opsdroid 0.19 has been released, not many matrix specific features in this release, just making it slightly easier to get a connector or database instance by name. However, this release paves the way for the merge of the pull request transitioning opsdroid to use the matrix-nio Python library, so the next release should be packed full of matrix stuff 😀

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

That's all I know 🏁

Thanks to Alexander and Nico for their help editing this edition.

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.15.1 is a bug fix release to fix two bugs introduced in 1.15.0.

The first prevented users of certain clients (including Riot) to add or remove third party identifiers such as email addresses.

The second caused Synapse not to start when configured with certain password auth providers.

Sorry if you have been bitten!

Get 1.15.1 from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

Changelog since v1.15.0

Synapse 1.15.1 (2020-06-16)

Bugfixes

• Fix a bug introduced in v1.15.0 that would crash Synapse on start when using certain password auth providers. (#7684)
• Fix a bug introduced in v1.15.0 which meant that some 3PID management endpoints were not accessible on the correct URL. (#7685)

Hi all,

Regular readers of TWIM may be familiar with the Decentralized Systems and Network Services Research Group at Karlsruhe Institute of Technology, who have been busy over the last few years analysing Matrix from an independent academic point of view. The work started in 2018 with Florian Jacob’s DSN Traveler spidering project, resulting in the Glimpse of the Matrix paper analysing Matrix’s scale and room/server distribution (at least as it was back then).

Last week, they released an entirely new paper: Matrix Decomposition: Analysis of an Access Control Approach on Transaction-based DAGs without Finality by Florian Jacob, Luca Becker, Jan Grashöfer and Hannes Hartenstein, presented at ACM SACMAT ‘20.

Now, the new paper is an absolutely fascinating deep dive analysis into State Resolution v2 - the algorithm at the heart of Matrix which defines how servers merge together their potentially conflicting copies of a given room, such that everyone ends up eventually with a consistent view… even in the face of bad actors. This means that Matrix effectively implements a decentralised access control system - ensuring that users stay banned, and only users with permission can ban, etc. You can see the slides below, and read the full paper here. The video of Florian’s talk from SACMAT should be published shortly.

To give some context from the Matrix side: designing and implementing State Resolution v2 back in 2018 was a bit of a mission. Our original v1 implementation had some bugs which meant that the result of the merge could unexpectedly favour historical state over the current state (so called ‘state resets’) - thus giving an attacker a way to maliciously revert the state of the room. In v2 we thought much more carefully about the algorithm, considering state present in one version of the room but not the other as a conflict, separating and applying access control events from regular events, and adding additional ordering of the state in the room by considering events in the context of their authorisation chain (the ‘auth DAG’). The end result is that we feel confident in v2 State Res, and we haven’t seen any problems with it in the wild since we shipped it in July 2018.

However: state resolution is not intuitive at first - for instance, when you merge two versions of a room together, you treat the state events as unordered sets… even though they are ordered in the context of the room DAG. The reason is that state res needs to work even if you don’t have a copy of the whole room DAG (otherwise you’d have to download way too much data to participate in a large room). Another example is the sequence in which orderings are then applied to the state events - and how that interacts with re-authorising those events, to stop malicious ones creeping in. In the core team, we’ve end up describing it several different ways to try to help folks understand: first Erik’s original MSC1442, then uhoreg’s literary Haskell implementation, then the terse reference version in the Spec itself, and most recently Neil Alexander’s State Resolution v2 for the Hopelessly Unmathematical.

As a result we are very excited and happy that Florian and the DSN team have now published the first ever independent in-depth analysis of the algorithm, particularly in the context of decentralised access control (i.e. enforcing bans, power levels, etc). We’re pleasantly surprised that apparently “To the best of our knowledge, Matrix is the only system that implements access control based on an eventually consistent partial order without finality and without a consensus algorithm”.

Even better, the DSN team found some remaining thinkos in Synapse’s implementation and the Matrix specification, which could have caused resolution results to diverge from other implementations, specifically:

1. we weren’t enforcing integers in JSON to be within range [-2⁵³+1, 2⁵³-1], fixed in https://github.com/matrix-org/synapse/pull/7381 and MSC2540
2. we forgot to include the notification field when authing power level events, fixed in https://github.com/matrix-org/synapse/issues/7501 and MSC2209 (thanks to Luca from DSN for the MSC!)
3. we forgot to spec the limit that one should apply to the number of parents of an event in the DAG (fixed in https://github.com/matrix-org/matrix-doc/pull/2538)
4. we missed that moderators could set server ACLs which could let them undermine room admins (fixed in https://github.com/matrix-org/synapse/pull/6834).

All of these have now been fixed in Synapse and the latest versions of the spec (room v6), and we’d like to sincerely thank Florian and Luca for rapidly and responsibly disclosing the issues to us. In other words: this research is directly improving Matrix, and it’s even more exciting that the stated future work for the DSN team is to work on a formal verification for the security of Matrix’s authorisation rules and state resolution. This stuff is tough, as anyone who’s played with TLA⁺ will know, and we are incredibly glad that the research community is helping out to formalise and hopefully prove that State Res v2 is as good as we think it is.

We should stress that DSN’s work is completely independent of The Matrix.org Foundation or anyone else building on the protocol; we’re just writing about it here because we think it’s incredibly cool and deserves the attention of the whole Matrix ecosystem.

Thanks again to Florian and the team - we look forward to seeing what comes next!

Open Tech Will Save Us #3 🎙

If you somehow missed this, please take a look!

• 1:00 Rabble talking about Planetary and SSB
• 31:19 Annie on Ditto
• 1:10:20 Erik on Synapse Perf

Dept of Spec 📜

Spec

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• MSC2265: Proposal for mandating case folding when processing e-mail address localparts

MSCs in Final Comment Period:

• MSC2399: Reporting that decryption keys are withheld (merge)

New MSCs:

• MSC2631: Add fallback_content to PusherData

• MSC2630: checking public keys in SAS verification

• MSC2625: Add mark_unread push rule action

• MSC2618: Helping others with mandatory implementation guides

Spec Core Team

In terms of Spec Core Team MSC focus for this week, we're sticking with the same three from last week: MSC2366 (verification flows), MSC2403 (knocking), and MSC2630 (SAS security).

In possibly more exciting news, the Spec Core Team has decided to start separating the scattered casual implementation tips and notes in the spec out and putting them in a series of implementation guides! These will take the form of gitbook-style things, and will initially feature front-to-back implementation guides for a generic Matrix client and homeserver. For more details to provide feedback, check out the associated MSC2618!

Dept of GSoC 🎓

This is the second week of the coding phase of GSoC 2020. Find updates from the students below:

GSOC/Opsdroid

tyagdit told us:

• The port from matrix-python-sdk to matrix-nio is almost completed. Sending and receiving most events works properly. Changes are expected be merged in a few days. This PR itself doesn't implement E2EE, that will be a different PR

• Alongside that I've been working on implementing E2EE with nio which is partially done, sending and receiving messages works so far. some changes to the MatrixStore interaction and testing is left.

Nheko

Chethan told us:

Device verification for Nheko

• Done
  • Adding Device List ( Although it already exists it is re-written in qml )
  • Detecting to_device device_verification messages
  • Part of the flow is done
• Yet to Complete
  • Complete the rest of the flow

go-neb

nikofil offered:

• Implementing end-to-end encryption for go-neb turns out to be a lot easier than expected thanks to the https://github.com/tulir/mautrix library, which contains out of the box e2ee support!

• The first week of GSoC was spent switching go-neb's usage of gomatrix for mautrix, which was easy as the latter is a fork of the former. This caused a pretty big PR that touched most of go-neb's files, although the changes were simple and easy to review. The PR was merged shortly after into go-neb.

• During the second week, it was time to add some basic e2ee support to go-neb. This was complicated slightly due to the bot's ability to create multiple clients and its treatment of /sync responses, however in the end e2ee is now working.

• The code is still of course a mess and session keys are often lost in the void. The code will be cleaned up and tested further next week, bugs will be (hopefully) fixed and another PR will be submitted to go-neb then to officially add e2ee support!

• Code can be found at https://github.com/nikofil/go-neb/tree/e2ee until it can be cleaned up

HTML-Embeddable rooms

From arnav-t:

Matrix has enormous potential to be used in many different use cases be it embedded chat rooms, comments section, chat boxes, etc. This project aims to fill that niche by providing an easy to use and easy to deploy client for users to embed on their webpages.
The client would mesh well wherever it is embedded by providing a postMessage interface to allow the parent webpage to interact with the client and dynamically modify it whenever needed (For example:- A comments section that is always the same theme as its parent webpage).

Features

As of now, the following features have been added (with more on the way)

• List of joined rooms that can be selected
• Live room timeline events
• Support for image-based messages
• Message composer and ability to send messages to a room
• Dark and light themes for the client
• Changeable highlight colors
• Toggleable room header, room timeline, and message composer components
• postMessage interface for sending commands from the parent window

Currently, I'm working on adding more options to the postMessage interface as well as adding more interactivity to the client itself.

Find out more on the project page: https://github.com/arnav-t/riot-embedded.

Ruma/Matrix GSoC update

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

devinr528 announced:

This week in ruma-events' Google Summer of Code project, I was able to finish the macros needed to generate the event content enums and trait implementations required for events. I started by defining the generic event structs (state, message, etc.) and manually writing the Serialize and Deserialize implementations. Over the next few days, this was moved into a custom derive macro called Event. The derive now implements all necessary traits with appropriate bounds, so a StateEvent<C> can not contain any ephemeral event content and so forth. I have removed the raw mod and related FromRaw and TryFromRaw traits, moving the validation into the deserialization and constructor for the few types that needed it. On the event content side of things, a function like procedural macro was used to allow declaring the enum using Matrix event type identifiers.

event_content_enum! {

    /// Any message event.
    name: AnyMessageEventContent,

    events: [
        "m.call.answer",

        "m.room.message",
        // ...

    ]
}

// Produces
pub enum AnyMessageEventContent {

    CallAnswer(CallAnswerEventContent),
    RoomMessage(MessageEventContent),

}

Dept of Servers 🏢

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander offered:

Much of this week's work has been maintenance-related with refactoring, cleaning up etc.

• Key fetching/expiry behaviour has been fixed, improving the reliability of federation.

• The monolith and component setup has now been refactored.

• A number of unused internal APIs have now been removed.

• Other internal roomserver APIs have been streamlined.

We have also started building an Yggdrasil-powered P2P demo (as opposed to the previous demos built using libp2p) and even includes support for embedding Riot Web in the single binary. It's not very stable/usable yet but there will be more news on that to follow within the next couple of weeks.

We have also started building and releasing Docker images for Dendrite on Docker Hub for those who want to use Docker to test Dendrite deployment.

Spec compliance is unchanged from last week:

• Client-Server APIs: 36%

• Server-Server APIs: 35%

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo reported:

Welcome back! This week I focused my efforts on better error handling. My goal was that every problem would be forwarded to the client (e.g. InternalServerError response) and also be logged. Yesterday I merged the giant PR this resulted in. Hopefully this will make Conduit easier to debug in the future.

Here are some other things I finished this week:

• User-interactive authentication (e.g. you need to type your password again if you want to delete devices)

• Config option to disable registration

I also found the problem with Riot not showing notifications for new messages and created a bug report. With a few tweaks, I managed to solve this on my account and now mentions, DMs and more play the notification sound (it also works on Riot X!).

@gnieto fixed a bug that prevented Riot from loading history when you join a room and @PublicByte implemented the /whoami endpoint.

Thanks to my supporters on Liberapay!

Synapse

Neil said:

This week we shipped 1.15.0, edited highlights include:

Features

• Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. (#7481)

• Add an option to disable autojoining rooms for guest accounts. (#6637)

• For SAML authentication, add the ability to pass email addresses to be added to new users' accounts via SAML attributes. Contributed by Christopher Cooper. (#7385)

• Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. Contributed by @WGH-. (#7586)

• Support the standardized m.login.sso user-interactive authentication flow. (#7630)

Bugfixes

• Allow new users to be registered via the admin API even if the monthly active user limit has been reached. Contributed by @dklimpel. (#7263)

• Fix a bug in automatic user creation during first time login with m.login.jwt. Regression in v1.6.0. Contributed by @olof. (#7585)

• Fix a bug causing the cross-signing keys to be ignored when resyncing a device list. (#7594)

• Fix bug where returning rooms for a group would fail if it included a room that the server was not in. (#7599)

• Fix duplicate key violation when persisting read markers. (#7607)

• Prevent an entire iteration of the device list resync loop from failing if one server responds with a malformed result. (#7609)

• Pass device information through to the login endpoint when using the login fallback. (#7629)

• Advertise the m.login.token login flow when OpenID Connect is enabled. (#7631)

• Fix bug in account data replication stream. (#7656)

Internal Changes

• Improve query performance for fetching state from a PostgreSQL database. Contributed by @ilmari. (#7567)

• Speed up processing of federation stream RDATA rows. (#7584)

• Refactor Ratelimiter to limit the amount of expensive config value accesses. (#7595)

• Clean up exception handling in SAML2ResponseResource. (#7614)

• Check if the localpart of a Matrix ID is reserved for guest users earlier in the registration flow, as well as when responding to requests to /register/available. (#7625)

Aside from that we continue out push towards improving performance both in terms of sharding workers but also improving tools to manage disk space usage. Specifically this week and next we are looking at db tuning and sharding the media repo worker. We are also noticing that the typing handler is chewing more CPU than would be expected. It could be a good candidate for moving away from the master process.

More broadly we are changing the defaults for notifications and making changes to support a more general overhaul of notifications.

Finally we are looking at how to not only delete messages after a certain time period, but also how to delete the associated media (which becomes difficult when considering e2e content along side long lived content such as avatars and sticker packs).

For more detail on what is coming next, take a look at our public task board.

Synapse Deployment 📥

Kubernetes

Ananace told us:

Just pushed version 1.15.0 of the K8s-optimized Synapse images

Dept of Bridges 🌉

mautrix-whatsapp

Tulir told us:

The WhatsApp bridge can now bridge location messages from WhatsApp to m.location on Matrix and contact messages as vCard files (until Matrix gets a native contact message type).

It also now encrypts media properly when bridging messages in encrypted rooms. I had apparently missed that when adding end-to-bridge encryption earlier.

mautrix-facebook

Tulir said:

mautrix-facebook got a few improvements and bugfixes:

• Added timestamp massaging

• Added support for bridging video and file messages to Matrix

• Fixed backfilling with SQLite (it didn't work due to the python timezone mess, fix contributed by erdnaxeli)

• Improved options for automatic reconnecting. The bridge should now work fairly reliably when using all the auto-reconnect options and backfilling, it only breaks if facebook decides you've been hacked

mautrix bridges in general

Tulir said:

Since MSC2190 and the related Synapse PR aren't moving much, all my bridges now generate appservice registration files with the workaround. The workaround is needed to avoid patching synapse or using other hacks, and I decided to enable it by default as many people had problems related to having to apply it manually.

The WhatsApp, Facebook and Hangouts bridges now have an option to use double puppeting to disable notifications when backfilling. It can be useful if you want to copy chat history to Matrix, but don't want to be spammed with hundreds of notifications while it's happening.

Nova-slack

eric offered:

To get things ready for NovaChat launch, we added a bunch of new features to a fork of the mx-puppet-slack client that Sorunome created. You can try it out here. We'll be pushing our changes upstream soon.

New features:

• message backfilling at room creation and reconnect

• syncs read status to/from Slack.com

• adds all users to bridged room during initial creation

• automatically bridges all channels and DMs that you are part of during initial setup, so you don't have to wait for a message to arrive for it to appear in Matrix

• tons of little reliability fixes

matrix-sms-bridge

Benedict announced:

My matrix-sms-bridge got an large update, which introduced an automated way to create and write messages to telephone numbers via commands in a bot room. See the docs for more informations, how to use it.

Dept of Clients 📱

NovaChat v0.1.1

eric reported:

Introducing NovaChat - a desktop chat client that supports 9+ different chat networks, built on top of Matrix. Think of it as a modern Adium/Trillian.

How it works

NovaChat is a system with three parts designed to work together:

• a new desktop client, based on Riot-web

• a federating homeserver (nova.chat)

• a set of Matrix bridges configured to run on a private server called a bridgebox

Matrix bridges are awesome but generally require you to run own homeserver. NovaChat removes that restriction, without sacrificing security and privacy. Your remote chat network credentials never leave your bridgebox, and messages are encrypted* on the bridgebox before being sent to the NovaChat homeserver.

*Encryption is currently enabled on Telegram/FB/Hangouts/Whatsapp bridges, mx-puppet-bridges coming shortly)

Very rough system diagram:

> NovaChat desktop client <> NovaChat homeserver <> bridgebox <> remote chat networks

Features

• Fancy new UI, heavily inspired by Telegram. See photo below or see it in action

• Easy graphical bridge set up (no more fiddling with config files)

• Reliable bridging, with delivery confirmations and bridge disconnect warnings

• Includes 8 pre-configured bridges: Facebook/Whatsapp/Hangouts/Telegram/Slack/Instagram/Twitter/Skype. Planning to add more soon, or submit a PR!

• Works on Mac OS, Linux and Windows

You don't need to be technical to use NovaChat, it's designed to Just WorkTM. If you need help setting up a bridgebox, we are offering hosting and maintenance for USD $5 per month.

The vast majority of this software has been created by the talented tulir and dm0141. I personally am a terrible coder (you can tell which parts I did), but I do have experience building products and teams, and I'm excited to begin working in the Matrix universe. I wrote recently about how I believe Matrix is the Universal Communication Bus! Thanks goes to Sorunome for creating the mx-puppet-bridges!

We're developing the whole project in the open (though not all components are open source), check the issue backlog :)

Want to help? We're hiring Matrix developer to work on the project with us. Send me a message if you're interested! Particularly looking for a React dev to help with frontend on nova-web.

NovaChat chat room: #novachat:nova.chat

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

red_sky announced:

nheko 0.7.2 is ready for release. Here's the summary of changes:

Highlights

• Reactions

  • React to a message with an emoji! 🎉

  • Reactions are shown below a message in a small bubble with a counter.

  • By clicking on that, others can add to the reaction count.

  • It may help you celebrating a new Nheko Release or react with a 👎 to a failed build to express your frustration.

  • This uses a new emoji picker. The picker will be improved in the near future (better scrolling, sections, favorites, recently used or similar) and then probably replace the current picker.

• Support for tagging rooms [tag]

  • Assign custom tags to rooms from the context menu in the room list.

  • This allows filtering rooms via the group list. This puts you in a focus mode showing only the selected tags.

  • You can assign multiple tags to group rooms however you like.

• SSO Login

  • With this you can now login on servers, that only provide SSO.

  • Just enter any mxid on the server. Nheko will figure out that you need to use SSO and redirect your browser to the login page.

  • Complete the login in your browser and Nheko should automatically log you in.

Features

• Support for showing users presence and status message as well as setting your own status message"

• Respect exif rotation of images

• An italian translation (contributed by Lorenzo Ancora)

• Optional alerts in your taskbar (contributed by z33ky)

• Optional bigger emoji only messages in the timeline (contributed by lkito)

• Optional hover feedback on messages (contributed by lkito)

• /roomnick to change your displayname in a single room.

• Preliminary support for showing inline images.

• Warn about unencrypted messages in encrypted rooms.

Improvements

• perf: Use less CPU to sort the room list.

• Limit size of replies. This currently looks a bit rough, but should improve in the future with a gradient or at some other transition.

• perf: Only clean out old messages from the database every 500 syncs. (There is usually more than one sync every second)

• Improve the login and register masks a bit with hints and validation.

• Descriptions for settings (contributed by lkito)

• A visual indicator, that nheko is fetching messages and improved scrolling (contributed by Lasath Fernando)

Bugfixes

• Fix not being able to join rooms

• Fix scale factor setting

• Buildfixes against gcc10 and Qt5.15 (missing includes)

• Settings now apply immediately again after changing them (only exception should be the scale factor)

• Join messages should never have empty texts now

• Timeline should now fail to render less often on platforms with native sibling windows.

• Don't rescale images on every frame on highdpi screens.

Matrix-Static release v0.3.0

Michael (t3chguy) reported:

New Matrix-Static release v0.3.0

With improved support for media and tombstone events and a bunch of bug fixes! https://github.com/matrix-org/matrix-static/releases

Should be deployed to view.matrix.org shortly

Arch Linux package of radical-native

Peter Roberts told us:

I've made an Arch Linux package of radical-native.

Riot X

valere announced:

• Several improvements on SSO login flow 🔒

• Support of power levels ⚡️

• Add remove favorites 🌟

• Multiple bugs and glitches fixes 🐛

The team is currently actively working on:

• Voip 1:1 Calls 📞

• Cross Signing bootstrap simplification 🔑

• Performances Improvement 🔥

• More room settings ⚙️

Riot Web

Ryan told us:

There's been lots of work many fronts this week, including:

• Continued work on cross-signing bootstrapping robustness

• New room list implementation in progress

• Upgraded fonts for new emoji

• A refreshed look for matrix.to is underway

• The appearance tab is being tweaked and expanded

• Read marker consistency improvements

Most of this work is still in progress or has only landed on develop so far. Our next RC will happen next week.

Riot-iOS

Manu offered:

This week, we continued to iterate on the implementation of the new push notification service thanks to user feedbacks. Those feedbacks also show that the app uses less battery.

On login, this is now possible to recover your cross-signing and your key backup from your recovery passphrase or key.

Dept of SDKs and Frameworks 🧰

libolm 3.1.5

uhoreg told us:

libolm 3.1.5 has been released. It mainly collects the various build system and documentation fixes that have accumulated since the last release. It also includes a new wasm build target (which apparently is used for running matrix-rust-sdk in the browser), as well as TypeScript definitions for the JavaScript bindings.

Ruma

Ruma is a Rust project to create a comprehensive set of APIs for Matrix. Previously there was a Ruma homeserver project.

jplatte told us:

Most of the work this past week was on ruma-events, for more on that see the GSoC section! Devin got far enough with that so we now know it all works out and can replace the previous ruma-events API. When that was clear, ruma-events was also merged into our new monorepo; now the only remaining crate to be merged is ruma-client (where I'm also waiting for some pending changes).

Dept of Ops 🛠

Matrix Helm chart

Kubernetes Helm chart for easily deploying a Matrix homeserver stack

Typo Kign told us:

Version 2 of my Matrix Helm chart is out! It now allows for running a federating homeserver without needing an external load balancer for the federation service, potentially lowering costs (depending on your cloud provider), and definitely reducing the headache of managing TLS certificates!

dacruz21/matrix-chart is a Kubernetes Helm chart for easily deploying a Matrix homeserver stack, complete with Synapse, Riot, VoIP calls, bridges, and other backing services. It's ideal for anyone looking to run a large homeserver, or for corporate environments where security is paramount!

Dept of Internet of Things 💡

Tuple Weather Station Example

Alexander Eisele announced:

This week I've programmed a small POC for a Matrix weather station.

It's build using a ESP8266 Microcontroller and a BME280 Sensor for temparature, humidity and barometric pressure.

The data is send over the Tuple Matrix <-> MQTT bridge.

Code: https://github.com/derEisele/tuple-weather-example

You can join this project on #tuple:eiselecloud.de.

I'm personally really excited about this project!

Dept of Bots 🤖

Matrix bot for Redmine

progserega told us:

https://github.com/progserega/matrix_redmine_bot

1. Bot can create, accept (to user) new issues in Redmine from matrix

2. Bot can add comment or/and files to issue by it id

3. Bot can show messages (for user) from Redmine in matrix room

Dept of Guides 🧭

Levans said:

Anyone running a federating instance of Synapse will likely have seen this: Synapse is database-hungry. It tends to take a lot of space. In this post, I'm documenting how I shrunk my homeserver database from 100GB to a little under 8GB, during a long maintenance cleanup. https://levans.fr/shrink-synapse-database.html

Very pleased to see this, I have a feeling I'm going to be running it in the near future...

New Public Rooms 🏟

New room "Matrix Outreach"

JulianF reported:

There's a new room "Matrix Outreach" #matrix-outreach:matrix.org for folks interested in "Helping each other explain and promote Matrix to the public: creating slogans, introduction leaflets, info-graphics, elevator pitch, Wikipedia articles, and so on."

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

Synapse 1.15.0 release day folks.

We continue our push to improve performance across the board. Factoring out event persistence into a separate worker pointed the way to a host of small but collectively important improvements. Expect further changes in coming releases.

Aside from that the admin API continues to grow (this time device management) and we have improved device list syncing to aid e2ee reliability.

We are receiving an increasing number of high quality PRs from the community, please keep them coming. Special thanks to cg505, dklimpel, WGH, olof and ilmari

Get 1.15.0 from github or any of the sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md.

Changelog since v1.14.0

Synapse 1.15.0 (2020-06-11)

No significant changes.

Synapse 1.15.0rc1 (2020-06-09)

Features

• Advertise support for Client-Server API r0.6.0 and remove related unstable feature flags. (#6585)
• Add an option to disable autojoining rooms for guest accounts. (#6637)
• For SAML authentication, add the ability to pass email addresses to be added to new users' accounts via SAML attributes. Contributed by Christopher Cooper. (#7385)
• Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. (#7481)
• Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. Contributed by @WGH-. (#7586)
• Support the standardized m.login.sso user-interactive authentication flow. (#7630)

Bugfixes

• Allow new users to be registered via the admin API even if the monthly active user limit has been reached. Contributed by @dklimpel. (#7263)
• Fix email notifications not being enabled for new users when created via the Admin API. (#7267)
• Fix str placeholders in an instance of PrepareDatabaseException. Introduced in Synapse v1.8.0. (#7575)
• Fix a bug in automatic user creation during first time login with m.login.jwt. Regression in v1.6.0. Contributed by @olof. (#7585)
• Fix a bug causing the cross-signing keys to be ignored when resyncing a device list. (#7594)
• Fix metrics failing when there is a large number of active background processes. (#7597)
• Fix bug where returning rooms for a group would fail if it included a room that the server was not in. (#7599)
• Fix duplicate key violation when persisting read markers. (#7607)
• Prevent an entire iteration of the device list resync loop from failing if one server responds with a malformed result. (#7609)
• Fix exceptions when fetching events from a remote host fails. (#7622)
• Make synctl restart start synapse if it wasn't running. (#7624)
• Pass device information through to the login endpoint when using the login fallback. (#7629)
• Advertise the m.login.token login flow when OpenID Connect is enabled. (#7631)
• Fix bug in account data replication stream. (#7656)

Improved Documentation

• Update the OpenBSD installation instructions. (#7587)
• Advertise Python 3.8 support in setup.py. (#7602)
• Add a link to #synapse:matrix.org in the troubleshooting section of the README. (#7603)
• Clarifications to the admin api documentation. (#7647)

Internal Changes

• Convert the identity handler to async/await. (#7561)
• Improve query performance for fetching state from a PostgreSQL database. Contributed by @ilmari. (#7567)
• Speed up processing of federation stream RDATA rows. (#7584)
• Add comment to systemd example to show postgresql dependency. (#7591)
• Refactor Ratelimiter to limit the amount of expensive config value accesses. (#7595)
• Convert groups handlers to async/await. (#7600)
• Clean up exception handling in SAML2ResponseResource. (#7614)
• Check that all asynchronous tasks succeed and general cleanup of MonthlyActiveUsersTestCase and TestMauLimit. (#7619)
• Convert get_user_id_by_threepid to async/await. (#7620)
• Switch to upstream dh-virtualenv rather than our fork for Debian package builds. (#7621)
• Update CI scripts to check the number in the newsfile fragment. (#7623)
• Check if the localpart of a Matrix ID is reserved for guest users earlier in the registration flow, as well as when responding to requests to /register/available. (#7625)
• Minor cleanups to OpenID Connect integration. (#7628)
• Attempt to fix flaky test: PhoneHomeStatsTestCase.test_performance_100. (#7634)
• Fix typos of m.olm.curve25519-aes-sha2 and m.megolm.v1.aes-sha2 in comments, test files. (#7637)
• Convert user directory, state deltas, and stats handlers to async/await. (#7640)
• Remove some unused constants. (#7644)
• Fix type information on assert_*_is_admin methods. (#7645)
• Convert registration handler to async/await. (#7649)