mirror of
https://gitlab.com/jeancf/twoot.git
synced 2024-12-18 00:00:17 +00:00
182 lines
8.8 KiB
Markdown
182 lines
8.8 KiB
Markdown
# Twoot
|
|
|
|
Twoot is a python script that mirrors tweets from a twitter account to a Mastodon account.
|
|
It is simple to set-up on a local machine, configurable and feature-rich.
|
|
|
|
**14 JUL 2023** VERSION 4.2
|
|
|
|
Twoot can now handle threads. All tweets can again be uploaded on Mastodon. Tweets in a threads are
|
|
displayed in reverse chronological order in the main timeline (first tweet on top) to improve readability.
|
|
|
|
*When several toots are posted in the same run of toot it is possible that these toots do not appear in
|
|
chronological order on the timeline. If it is the case, try setting `upload_pause` to 3-5 seconds in
|
|
your config file to slow down the rate at which toots are uploaded.*
|
|
|
|
A list of nitter instances to use can now be specified in the config file
|
|
e.g. `nitter_instances = ["nitter.nl", "nitter.fdn.fr"]`.
|
|
If none is specified, the built-in list of 2-3 known good instances is used as before.
|
|
|
|
> Previous updates can be found in CHANGELOG.
|
|
|
|
## Features
|
|
|
|
* Fetch timeline of given user from twitter.com (through nitter instance)
|
|
* Scrape html and format tweets for post on mastodon
|
|
* Emojis supported
|
|
* Upload images from tweet to Mastodon
|
|
* Optionally upload videos from tweet to Mastodon
|
|
* Specify maximum age of tweet to be considered
|
|
* Specify minimum delay before considering a tweet for upload
|
|
* Remember tweets already tooted to prevent double posting
|
|
* Optionally update avatar and banner pictures on profile if changed
|
|
* Optionally post reply-to tweets on the mastodon account
|
|
* Optionally ignore retweets
|
|
* Optionally remove redirections (e.g. reveal destination of short URLs)
|
|
* Optionally remove trackers (UTM parameters) from URLs
|
|
* Optional domain substitution for Twitter, Youtube and Reddit domains (e.g. [Nitter](https://github.com/zedeus/nitter/wiki/Instances),
|
|
[Invidious](https://redirect.invidious.io/) and [teddit](https://teddit.net/) respectively)
|
|
* option to add timestamp of the original tweet to bottom of toot
|
|
* Optional footer line to add tags at bottom of toot
|
|
* Allows rate-limiting posts to Mastodon instance
|
|
|
|
## Usage
|
|
|
|
```sh
|
|
usage: twoot.py [-h] [-f <.toml config file>] [-t <twitter account>] [-i <mastodon instance>]
|
|
[-m <mastodon account>] [-p <mastodon password>] [-r] [-s] [-l] [-u] [-v] [-o] [-q]
|
|
[-a <max age (in days)>] [-d <min delay (in mins)>] [-c <max # of toots to post>]
|
|
```
|
|
|
|
## Arguments
|
|
|
|
Assuming that the Twitter handle is @SuperDuperBot and the Mastodon account
|
|
is `sd@example.com` on instance masto.space:
|
|
|
|
|Switch |Description | Example | Required |
|
|
|-------|--------------------------------------------------|--------------------|--------------------|
|
|
| -f | path of `.toml` file with configuration | `SuperDuper.toml` | No |
|
|
| -t | twitter account name without '@' | `SuperDuper` | If no config file |
|
|
| -i | Mastodon instance domain name | `masto.space` | If no config file |
|
|
| -m | Mastodon username | `sd@example.com` | If no config file |
|
|
| -p | Mastodon password | `my_Sup3r-S4f3*pw` | Once at first run |
|
|
| -v | Upload videos to Mastodon | *N/A* | No |
|
|
| -o | Do not add "Original tweet" line | *N/A* | No |
|
|
| -q | Update avatar and banner on profile if changed | *N/A* | No |
|
|
| -r | Post reply-to tweets (ignored by default) | *N/A* | No |
|
|
| -s | Skip retweets (posted by default) | *N/A* | No |
|
|
| -l | Remove link redirections | *N/A* | No |
|
|
| -u | Remove trackers from URLs | *N/A* | No |
|
|
| -a | Max. age of tweet to post (in days) | `5` | No |
|
|
| -d | Min. age before posting new tweet (in minutes) | `15` | No |
|
|
| -c | Max number of toots allowed to post (cap) | `1` | No |
|
|
|
|
## Notes
|
|
|
|
### Password
|
|
|
|
A password must be provided for the first run only. Once twoot has connected successfully to the
|
|
Mastodon host, an access token is saved in a `.secret` file named after the mastodon account,
|
|
and a password is no longer necessary (command-line switch `-p` is not longer required).
|
|
|
|
### Config file
|
|
|
|
A `default.toml` file is provided to be used as template. If `-f` is used to specify a config file
|
|
to use, all the other command-line parameters are ignored, except `-p` (password) if provided.
|
|
|
|
### Removing redirected links
|
|
|
|
`-l` (or `remove_link_redirections = true` in toml file) will follow every link included in the
|
|
tweet and replace them with the url that the resource is directly dowmnloaded from (if applicable).
|
|
e.g. bit.ly/xxyyyzz -> example.com
|
|
Every link visit can take up to 5 sec (timeout) depending on the responsiveness of the source
|
|
therefore this option will slow down tweet processing.
|
|
|
|
If you are interested by tracker removal (`-u`, `remove_trackers_from_urls = true`) you should
|
|
also select redirection removal as trackers are often hidden behind the redirection of a short URL.
|
|
|
|
### Uploading videos
|
|
|
|
When using the `-v` (`upload_videos = true`) switch consider:
|
|
|
|
* whether the copyright of the content that you want to cross-post allows it
|
|
* the storage / transfer limitations of the Mastodon instance that you are posting to
|
|
* the upstream bandwidth that you may consume on your internet connection
|
|
|
|
### Updating profile
|
|
|
|
If `-q` (`update_profile = true`) is specified, twoot will check if the avatar and banner pictures
|
|
have changed on the twitter page. This check compares the name of files used by twitter with the names
|
|
of the files that have been uploaded on Mastodon and if they differ both files are downloaded from
|
|
twitter and uploaded on Mastodon. The check is very fast if there is no update.
|
|
|
|
### Adding timestamp of original tweet to toot
|
|
|
|
Use `tweet_time_format` option in configuration file to specify the datetime format to display the date
|
|
at which the tweet was published next to the "Original tweet" link. Valid format specifiers are
|
|
the same as those used to format datetimes in python
|
|
(<https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>).
|
|
e.g. `tweet_time_format = "(%d %b %Y %H:%M %Z)"`
|
|
|
|
An empty or missing `tweet_time_format` disables the display of the timestamp.
|
|
|
|
By default, dates are specified in UTC time zone. To convert the timestamp to another time zone,
|
|
use the `tweet_timezone` option in configuration file. Valid time zone names are those of the Olson time
|
|
zone database (<https://en.wikipedia.org/wiki/Tz_database>)
|
|
e.g. `tweet_timezone = "Europe/Paris"`
|
|
|
|
### Rate control
|
|
|
|
Default max age is 1 day. Decimal values are OK.
|
|
|
|
Default min delay is 0 minutes.
|
|
|
|
No limitation is applied to the number of toots uploaded if `-c` is not specified.
|
|
|
|
## Installation
|
|
|
|
Make sure python3 is installed.
|
|
|
|
Twoot depends on `requests`, `beautifulsoup4`, `Mastodon.py` and `pytz` python modules.
|
|
Additionally, if you are using a version of python < 3.11 you also need to install the `tomli` module.
|
|
|
|
**Only If you plan to download videos** with the `-v` switch, are additional dependencies required:
|
|
|
|
* Python module `youtube-dl2`
|
|
* [ffmpeg](https://ffmpeg.org/download.html) (installed with the package manager of your distribution)
|
|
|
|
```sh
|
|
pip install beautifulsoup4 Mastodon.py youtube-dl2 pytz
|
|
```
|
|
|
|
In your user folder, execute `git clone https://gitlab.com/jeancf/twoot.git`
|
|
to clone repo with twoot.py script.
|
|
|
|
Add command line to crontab. For example, to run every 15 minutes starting at minute 1 of every hour
|
|
and process the tweets posted in the last 5 days but at least 15 minutes
|
|
ago:
|
|
|
|
```crontab
|
|
1-59/15 * * * * /path/to/twoot.py -t SuperDuper -i masto.space -m sd@example.com -p my_Sup3r-S4f3*pw -a 5 -d 15
|
|
```
|
|
|
|
## Featured Accounts
|
|
|
|
Twoot is known to be used for the following feeds (older first):
|
|
|
|
* [@todayilearned@noc.social](https://noc.social/@todayilearned)
|
|
* [@moznews@noc.social](https://noc.social/@moznews)
|
|
* [@hackster_io@noc.social](https://noc.social/@hackster_io)
|
|
* [@cnxsoft@noc.social](https://noc.social/@cnxsoft)
|
|
* [@unrealengine@noc.social](https://noc.social/@unrealengine)
|
|
* [@phoronix@noc.social](https://noc.social/@phoronix)
|
|
* [@uanews@fed.celp.de](https://fed.celp.de/@uanews)
|
|
* [@theregister@geeknews.chat](https://geeknews.chat/@theregister)
|
|
* [@arstechnica@geeknews.chat](https://geeknews.chat/@arstechnica)
|
|
* [@slashdot@geeknews.chat](https://geeknews.chat/@slashdot)
|
|
|
|
## Background
|
|
|
|
I started twoot when [tootbot](https://github.com/cquest/tootbot) stopped working.
|
|
Tootbot relied on RSS feeds from [https://twitrss.me](https://twitrss.me) that broke when Twitter
|
|
refreshed their web UI in July 2019.
|