mirror of
https://gitlab.com/jeancf/twoot.git
synced 2024-11-23 20:11:11 +00:00
185 lines
8.9 KiB
Markdown
185 lines
8.9 KiB
Markdown
# Twoot
|
|
|
|
Twoot is a python script that mirrors tweets from a twitter account to a Mastodon account.
|
|
It is simple to set-up on a local machine, configurable and feature-rich.
|
|
|
|
**24 JUL 2023** VERSION 4.3.2
|
|
|
|
Minor fixes (avoid interruption of processing):
|
|
|
|
* Re-add missing \n before url of quoted tweet
|
|
* Avoid interruption of processing in case of empty thread timeline
|
|
|
|
> Previous updates can be found in CHANGELOG.
|
|
|
|
## Features
|
|
|
|
* Fetch timeline of given user from twitter.com (through nitter instance)
|
|
* Scrape html and format tweets for post on mastodon
|
|
* Threads (series of replies to own messages) are replicated
|
|
* Emojis supported
|
|
* Upload images from tweet to Mastodon
|
|
* Optionally upload videos from tweet to Mastodon
|
|
* Specify maximum age of tweet to be considered
|
|
* Specify minimum delay before considering a tweet for upload
|
|
* Remember tweets already tooted to prevent double posting
|
|
* Optionally update avatar and banner pictures on profile if changed
|
|
* Optionally post reply-to tweets on the mastodon account
|
|
* Optionally ignore retweets
|
|
* Optionally remove redirections (e.g. reveal destination of short URLs)
|
|
* Optionally remove trackers (UTM parameters) from URLs
|
|
* Optional domain substitution for Twitter, Youtube and Reddit domains (e.g. [Nitter](https://github.com/zedeus/nitter/wiki/Instances),
|
|
[Invidious](https://redirect.invidious.io/) and [teddit](https://teddit.net/) respectively)
|
|
* option to add timestamp of the original tweet to bottom of toot
|
|
* Optional footer line to add tags at bottom of toot
|
|
* Allows rate-limiting posts to Mastodon instance
|
|
|
|
## Usage
|
|
|
|
```
|
|
usage: twoot.py [-h] [-f <.toml config file>] [-t <twitter account>] [-i <mastodon instance>]
|
|
[-m <mastodon account>] [-p <mastodon password>] [-r] [-s] [-l] [-u] [-v] [-o] [-q]
|
|
[-a <max age (in days)>] [-d <min delay (in mins)>] [-c <max # of toots to post>]
|
|
```
|
|
|
|
## Arguments
|
|
|
|
Assuming that the Twitter handle is @SuperDuperBot and the Mastodon account
|
|
is `sd@example.com` on instance masto.space:
|
|
|
|
|Switch |Description | Example | Required |
|
|
|-------|--------------------------------------------------|--------------------|--------------------|
|
|
| -f | path of `.toml` file with configuration | `SuperDuper.toml` | No |
|
|
| -t | twitter account name without '@' | `SuperDuper` | If no config file |
|
|
| -i | Mastodon instance domain name | `masto.space` | If no config file |
|
|
| -m | Mastodon username | `sd@example.com` | If no config file |
|
|
| -p | Mastodon password | `my_Sup3r-S4f3*pw` | Once at first run |
|
|
| -v | Upload videos to Mastodon | *N/A* | No |
|
|
| -o | Do not add "Original tweet" line | *N/A* | No |
|
|
| -q | Update avatar and banner on profile if changed | *N/A* | No |
|
|
| -r | Post reply-to tweets (ignored by default) | *N/A* | No |
|
|
| -s | Skip retweets (posted by default) | *N/A* | No |
|
|
| -l | Remove link redirections | *N/A* | No |
|
|
| -u | Remove trackers from URLs | *N/A* | No |
|
|
| -a | Max. age of tweet to post (in days) | `5` | No |
|
|
| -d | Min. age before posting new tweet (in minutes) | `15` | No |
|
|
| -c | Max number of toots allowed to post (cap) | `1` | No |
|
|
|
|
## Notes
|
|
|
|
### Password
|
|
|
|
A password must be provided for the first run only. Once twoot has connected successfully to the
|
|
Mastodon host, an access token is saved in a `.secret` file named after the mastodon account,
|
|
and a password is no longer necessary (command-line switch `-p` is not longer required).
|
|
|
|
### Config file
|
|
|
|
A `default.toml` file is provided to be used as template. If `-f` is used to specify a config file
|
|
to use, all the other command-line parameters are ignored, except `-p` (password) if provided.
|
|
|
|
### Removing redirected links
|
|
|
|
`remove_link_redirections = true` in toml file (or `-l` on the command line ) will follow every link
|
|
included in the tweet and replace them with the url that the resource is directly dowmnloaded from
|
|
(if applicable). e.g. bit.ly/xxyyyzz -> example.com
|
|
|
|
Every link visit can take up to 5 sec (timeout) depending on the responsiveness of the source
|
|
therefore this option will slow down tweet processing.
|
|
|
|
If you are interested by tracker removal (`remove_trackers_from_urls = true`, `-u`) you should
|
|
also select redirection removal as trackers are often hidden behind the redirection of a short URL.
|
|
|
|
### Uploading videos
|
|
|
|
When using the `upload_videos = true` (`-v`) switch consider:
|
|
|
|
* whether the copyright of the content that you want to cross-post allows it
|
|
* the storage / transfer limitations of the Mastodon instance that you are posting to
|
|
* the upstream bandwidth that you may consume on your internet connection
|
|
|
|
### Updating profile
|
|
|
|
If `update_profile = true` (`-q`) is specified, twoot will check if the avatar and banner pictures
|
|
have changed on the twitter page. This check compares the name of files used by twitter with the names
|
|
of the files that have been uploaded on Mastodon and if they differ both files are downloaded from
|
|
twitter and uploaded on Mastodon. The check is very fast if there is no update.
|
|
|
|
### Adding timestamp of original tweet to toot
|
|
|
|
Use `tweet_time_format` option in configuration file to specify the datetime format to display the date
|
|
at which the tweet was published next to the "Original tweet" link. Valid format specifiers are
|
|
the same as those used to format datetimes in python
|
|
(<https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>).
|
|
e.g. `tweet_time_format = "(%d %b %Y %H:%M %Z)"`
|
|
|
|
An empty or missing `tweet_time_format` disables the display of the timestamp.
|
|
|
|
By default, dates are specified in the local timezone of the machine running the script. To display the timestamp to another time zone, use the `tweet_timezone` option in configuration file. Valid time zone names are those of the Olson time
|
|
zone database (<https://en.wikipedia.org/wiki/Tz_database>).
|
|
e.g. `tweet_timezone = "Europe/Paris"` or `tweet_timezone = "UTC"`
|
|
|
|
### Rate control
|
|
|
|
Default max age is 1 day. Decimal values are OK.
|
|
|
|
Default min delay is 0 minutes.
|
|
|
|
No limitation is applied to the number of toots uploaded if `-c` is not specified.
|
|
|
|
If messages in a thread that are uploaded simultaneously appear in the wrong order, try setting
|
|
the `upload_pause` configuration variable in the configuration file to a few seconds (start with 3-5).
|
|
|
|
## Installation
|
|
|
|
Make sure python3 is installed.
|
|
|
|
Twoot depends on `requests`, `beautifulsoup4`, `Mastodon.py` and `pytz` python modules.
|
|
Additionally, if you are using a version of python < 3.11 you also need to install the `tomli` module.
|
|
|
|
**Only If you plan to download videos** with the `-v` switch, are additional dependencies required:
|
|
|
|
* Python module `youtube-dl2`
|
|
* [ffmpeg](https://ffmpeg.org/download.html) (installed with the package manager of your distribution)
|
|
|
|
```sh
|
|
pip install beautifulsoup4 Mastodon.py youtube-dl2 pytz
|
|
```
|
|
|
|
In your user folder, execute `git clone https://gitlab.com/jeancf/twoot.git`
|
|
to clone repo with twoot.py script.
|
|
|
|
If you want to use a config file to specify options (recommended), copy `default.toml` to
|
|
`[you_preferred_name].toml` and edit it to your preferences.
|
|
|
|
Add command line to crontab. For example, to run every 15 minutes starting at minute 1 of every hour
|
|
and process the tweets posted in the last 5 days but at least 15 minutes
|
|
ago:
|
|
|
|
```crontab
|
|
1-59/15 * * * * /path/to/twoot.py -t SuperDuper -i masto.space -m sd@example.com -p my_Sup3r-S4f3*pw -a 5 -d 15
|
|
```
|
|
|
|
After the first successful run, you no longer need to specify the password and yoiucan remove the `-p` switch.
|
|
|
|
## Featured Accounts
|
|
|
|
Twoot is known to be used for the following feeds (older first):
|
|
|
|
* [@todayilearned@noc.social](https://noc.social/@todayilearned)
|
|
* [@moznews@noc.social](https://noc.social/@moznews)
|
|
* [@hackster_io@noc.social](https://noc.social/@hackster_io)
|
|
* [@cnxsoft@noc.social](https://noc.social/@cnxsoft)
|
|
* [@unrealengine@noc.social](https://noc.social/@unrealengine)
|
|
* [@phoronix@noc.social](https://noc.social/@phoronix)
|
|
* [@uanews@fed.celp.de](https://fed.celp.de/@uanews)
|
|
* [@theregister@geeknews.chat](https://geeknews.chat/@theregister)
|
|
* [@arstechnica@geeknews.chat](https://geeknews.chat/@arstechnica)
|
|
* [@slashdot@geeknews.chat](https://geeknews.chat/@slashdot)
|
|
|
|
## Background
|
|
|
|
I started twoot when [tootbot](https://github.com/cquest/tootbot) stopped working.
|
|
Tootbot relied on RSS feeds from [https://twitrss.me](https://twitrss.me) that broke when Twitter
|
|
refreshed their web UI in July 2019.
|