GitHub/sdrtrunk
1
0
Fork 0
mirror of https://github.com/DSheirer/sdrtrunk.git synced 2026-11-13 00:57:31 -05:00
Code Issues Packages Projects Releases 2 Wiki Activity

Identifier Framework and Message/Event Processing Overhaul (#455)

Browse Source 
* WIP on new P25 message framer.

* WIP on new P25 message framer.

* Adds new 1/2 and 3/4 rate trellis decoders.  Still trying to chase
down why there is a sync pattern mismatch in the NID buffer queue.

* WIP - new p25 message framer works correctly now.  PDU's are dectected
and decoded using the 1/2 and 3/4 rate trellis decoders.  Packet
Sequences are assembled.  Need to add a new packet sequence message
factory that can detect Motorola Auto Registration Service (ARS)
formatted UDP/ID packets and SNDCP command packets.

* WIP - removes nullify on channel results queue.

* WIP - updates message framer for TSBK 1/2/3 processing.

* WIP removing P25 sync pattern and nid bits from all message parsers.

* WIP - merges identifier framework branch changes into this branch.

* WIP on updating link control words to new format and identifier framework.

* WIP - implements rest of link control words for extended formats

* WIP - initial work on ISP/OSP TSBK message restructuring.

* WIP - adds more TSBK ISP and OSP messages

* WIP - finished updating most of the TSBK ISP and OSP messages.  Converted
all messages to no longer include the NID (nac + dataunitID) so that
each message can simply hold the parsed values.  Removed AliasList as
an argument to each message and updated all messges to use the new
IIdentifer and IAPCO25Channel interfaces.

Removed legacy TDU and LDU link control messages and replaced with a
single LinkControlWord class hierarchy.

P25 decoder state is not yet updated for the new message structure.

PDU message parsing is not fully implemented.

* WIP - adds some Multi-Block trunking control messages with Alternate
and Unconfirmed formats.

* WIP - Fully implements AMBTC and UMBTC PDU message parsers.

Some package restructuring.

* WIP - Resolves Link control parsing in LDU1 and 2 messages.

* WIP - finishes updating TSBK and Link Control messages.  Adds support
for moto link control messages for patch groups.

* WIP - finishes updating TSBK and Link Control messages.  Adds support
for moto link control messages for patch groups.  Adds support for moto
ARS service, IPV4 decoding, and UDP/IP packets.

* WIP - finishes SNDCP packet data with header parsing.  Compressed IP or
UDP headers are not yet supported, but are logged as such.

* WIP - converts all talkgroup identifiers to integers and updates
alias list to create a per-protocol map of talkgroup id's to alias.

Inserts a legacy talkgroup (ltr, p25, passport, mpt1327, mdc, etc.)
conversion utility to convert existing legacy (string) talkgroups to
new integer talkgroup format.

Creates an audio packet manager to insert metadata into audio packets
before streaming packets for playback, recording, and/or streaming.

Creates identifier collection to manage current identifiers in each of
the channel and decoder states.

Creates simplified channel metadata to maintain the now playing display.

Moves channel attributes (alias list, system, site, name) as configuration
identifiers in the channel state.

Many other refactoring and updates.

* WIP - numerous updates to push use of identifier collection throughout
the system.  Updates the channel metadata and audio packet processing
logic.  Implements a large portion of the P25 decode state processing.
Updates traffic channel management.  Adds user preferences for timestamp
and identifier formatting, however doesn't add a user interface yet.

* WIP - initial implementation of user preferences editor window.

* WIP - updates preferences editor window to close and dispose window but
retain javafx event thread running.

* WIP - fully updates p25 decoder state and traffic channel processing to
use the new identifier framework.

Cleans up channel event processing and removes channel model as the sole
broadcaster of channel events.  Channel processing manager now listens
for all start/stop events and requestors dispatch event directly to the
channel processing manager.

Resolves issue in airspy that was causing user interface to lock-up.  The
issue appears to occur when the dongle is plugged into a usb port that
can't fully support the data rate.  sdrtrunk now recognizes these errors
and warns the user that they should use another usb port.

* WIP - updates alias action manager and adds new talkgroup range alias identifier.

Updates playlist to incorporate a version number so that the playlist
updater only runs when needed, to update a playlist to the correct
version format.

Updates user preferences to use the google event bus to broadcast
preference update notifications to remove the need for listeners to
register directly on the user preferences instance.

* WIP - updates alias id editors to flag invalid values and moves legacy
alias ids to legacy package.

Updates event logging configuration to add traffic channel decode events
and decoded message logging.  Removes binary message logging since that
is now replaced with demodulated bitstream recording.

* Removes system properties from user preferences.  Preferences are stored
in the OS-dependent settings store (windows - registry)

* Updates MapService to use new PlottableDecodeEvent instead of monitoring
all of the decoded messages.

Updates Fleetsync decoder and related messages to identifier framework.

* Updates LTR-Net, Fleetsync and LJ1200 decoders to new identifier
framework and adds support for user preferences for formatting of
LTR-Net and fleetsync talkgroup identifiers.

Breaks LTR-Net messages into various message types and uses a new
message factory.

Removes alias list from message processing for ltr-net, fleetsync and
lj1200.

* Updates LTR-Standard messages to Identifier framework.

* Removes Plottable and IPlottable from codebase.

* wip

* WIP on MPT-1327 messages.

* Updates MPT1327 decoder to identifier framework and new traffic channel
manager.  This decoder still needs to be updated with a new message
parser factory and a sync detector like what is now used for P25.

* Updates MDC1200 decoder and message to identifier framework.

* Updates Tait1200 and Passport decoders to identifier framework.

* Merges changes from master branch.

* Removes legacy alias support classes.  Updates passport message processor
to only pass valid messages.  Adds support to playlist updater for
converting passport mobile ids to become passport talkgroup values.

* Updates P25 decoder state and channel state to better track allocated
traffic channels.  Resolves issue in timestamp manager for trackig
message timestamps. Resolves issue in airspy tuner that was causing
the tuner to stop sending samples to the application.

* Update P25 Decoder state to process SNDCP management messages and
PDU messages.

Updates P25 activity summary to display network and site details.

Resolves issue with overlay panel that prevented enabled channels from
rendering correctly when zoomed in.

* Creates a new FilePreferences as part of the user preferences to begin
the transition of SystemProperties over to user preferences and consolidate
file preferences in one location.

Updates playlist manager to use the new file preferences and to use
default.xml as the new default playlist name, following conversion of
a playlist_v2.xml format to the new format.

In the near future, the user will be able to edit file locations and to
create/use other playlists.

Resolved an issue with all decoder states where not all identifiers were
being included in decode events.
This commit is contained in:
Denny Sheirer
2018-12-24 09:08:34 -05:00
committed by GitHub
parent 735fca16d6
commit b67d4d92dc
869 changed files with 64776 additions and 45233 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/main/java/io/github/dsheirer/edac/trellis/DMRNode.java → src/main/java/io/github/dsheirer/edac/trellis/DMR_3_4_Node.java
View File

@@ -18,7 +18,7 @@
******************************************************************************/
package io.github.dsheirer.edac.trellis;
public class DMRNode extends Node
public class DMR_3_4_Node extends Node
{
//Hamming error values indicating the number of bits set in each indexed value, 0-15
public static final int[] HAMMING_ERROR_COUNT = new int[]{0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4};
@@ -47,7 +47,7 @@ public class DMRNode extends Node
* @param inputValue for the trellis time instant
* @param transmittedOutputValue the actual four-bit output value that was transmitted.
*/
public DMRNode(int inputValue, int transmittedOutputValue)
public DMR_3_4_Node(int inputValue, int transmittedOutputValue)
{
super(inputValue, transmittedOutputValue);
}

85
src/main/java/io/github/dsheirer/edac/trellis/P25_1_2_Node.java Normal file
View File

@@ -0,0 +1,85 @@
/*******************************************************************************
* sdrtrunk
* Copyright (C) 2014-2017 Dennis Sheirer
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>
*
******************************************************************************/
package io.github.dsheirer.edac.trellis;
public class P25_1_2_Node extends Node
{
/**
* Hamming error values indicating the number of bits set in each indexed value, 0-15. XOR the
* transmitted 4-bit value against the reference 4-bit value to obtain the lookup index to this array.
*/
public static final int[] HAMMING_ERROR_COUNT = new int[]{0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4};
/**
* P25 Finite State Machine 1/2 Rate transition matrix.
*
* Note: these transmitted bit values are translated from TIA-102 BAAA Table 7-2 encoder state
* table constellation values.
*/
public static final int[][] TRANSITION_MATRIX = new int[][]
{
{2,12,1,15},//0,15,12,3
{14,0,13,3},//4,11,8,7
{9,7,10,4}, //13,2,1,14
{5,11,6,8} //9,6,5,10
};
/**
* P25 1/2-rate Trellis Coded Modulation (TCM) node
*
* @param inputValue for the trellis time instant
* @param transmittedOutputValue the actual four-bit output value that was transmitted.
*/
public P25_1_2_Node(int inputValue, int transmittedOutputValue)
{
super(inputValue, transmittedOutputValue);
}
/**
* Calculates the error between the preceeding node and this node as the Hamming distance between the expected
* four-bit output value versus the actual transmitted output value.
*
* @param precedingNode that precedes this node
* @return error value for this node relative to the preceeding node argument
*/
@Override
public int getError(Node precedingNode)
{
if(precedingNode != null)
{
int expectedOutputValue = getOutputValue(precedingNode);
int errorMask = expectedOutputValue ^ getTransmittedOutputValue();
return HAMMING_ERROR_COUNT[errorMask];
}
return 0;
}
/**
* Expected output four-bit value when this node is preceeded by the argument node.
*
* @param precedingNode that precedes this node
* @return expected output value
*/
@Override
public int getOutputValue(Node precedingNode)
{
return TRANSITION_MATRIX[precedingNode.getInputValue()][getInputValue()];
}
}

84
src/main/java/io/github/dsheirer/edac/trellis/P25_3_4_Node.java Normal file
View File

@@ -0,0 +1,84 @@
/*******************************************************************************
* sdrtrunk
* Copyright (C) 2014-2017 Dennis Sheirer
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>
*
******************************************************************************/
package io.github.dsheirer.edac.trellis;
public class P25_3_4_Node extends Node
{
//Hamming error values indicating the number of bits set in each indexed value, 0-15
public static final int[] HAMMING_ERROR_COUNT = new int[]{0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4};
/**
* Trellis encoder state transition table. Note: this table is converted from the ICD state
* transition table where the table symbol is converted to the transmitted bit value.
*/
public static final int[][] TRANSITION_MATRIX = new int[][]
{
{2,13,14,1,7,8,11,4},
{14,1,7,8,11,4,2,13},
{10,5,6,9,15,0,3,12},
{6,9,15,0,3,12,10,5},
{15,0,3,12,10,5,6,9},
{3,12,10,5,6,9,15,0},
{7,8,11,4,2,13,14,1},
{11,4,2,13,14,1,7,8}
};
/**
* Trellis Coded Modulation (TCM) 3/4-rate trellis node for decoding P25
*
* @param inputValue for the trellis time instant
* @param transmittedOutputValue the actual four-bit output value that was transmitted.
*/
public P25_3_4_Node(int inputValue, int transmittedOutputValue)
{
super(inputValue, transmittedOutputValue);
}
/**
* Calculates the error between the preceeding node and this node as the Hamming distance between the expected
* four-bit output value versus the actual transmitted output value.
*
* @param precedingNode that precedes this node
* @return error value for this node relative to the preceeding node argument
*/
@Override
public int getError(Node precedingNode)
{
if(precedingNode != null)
{
int expectedOutputValue = getOutputValue(precedingNode);
int errorMask = expectedOutputValue ^ getTransmittedOutputValue();
return HAMMING_ERROR_COUNT[errorMask];
}
return 0;
}
/**
* Expected output four-bit value when this node is preceeded by the argument node.
*
* @param precedingNode that precedes this node
* @return expected output value
*/
@Override
public int getOutputValue(Node precedingNode)
{
return TRANSITION_MATRIX[precedingNode.getInputValue()][getInputValue()];
}
}

154
src/main/java/io/github/dsheirer/edac/trellis/ViterbiDecoder_1_2_P25.java Normal file
View File

@@ -0,0 +1,154 @@
/*******************************************************************************
* sdrtrunk
* Copyright (C) 2014-2017 Dennis Sheirer
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>
*
******************************************************************************/
package io.github.dsheirer.edac.trellis;
import io.github.dsheirer.bits.BinaryMessage;
import io.github.dsheirer.bits.CorrectedBinaryMessage;
import java.util.List;
/**
* Viterbi decoder for APCO-25 1/2 rate Trellis Coded Modulation (TCM) encoded messages.
*/
public class ViterbiDecoder_1_2_P25 extends ViterbiDecoder
{
public ViterbiDecoder_1_2_P25()
{
super(2,4);
}
/**
* Decodes a 1/2 rate trellis coded modulation (TCM) encoded P25 binary message containing 196 bits that have
* already been deinterleaved.
*
* @param encodedMessage to decode that has already been deinterleaved.
* @return decoded message
*/
public CorrectedBinaryMessage decode(BinaryMessage encodedMessage)
{
int[] symbols = getSymbols(encodedMessage);
Path mostLikelyPath = decode(symbols);
return getMessage(mostLikelyPath);
}
/**
* Extracts the decoded/corrected message from the surviving path argument.
*
* @param path to extract a message from
* @return corrected binary message.
*/
private CorrectedBinaryMessage getMessage(Path path)
{
List<Node> mNodes = path.getNodes();
//Each node contains an input value excluding the starting(0) and final flushing(0) nodes.
CorrectedBinaryMessage message = new CorrectedBinaryMessage((mNodes.size() - 2) * getInputBitLength());
if(mNodes.size() > 2)
{
for(int x = 1; x < mNodes.size() - 1; x++)
{
int inputValue = mNodes.get(x).getInputValue();
int messageOffset = getInputBitLength() * (x - 1);
if((inputValue & 2) == 2)
{
message.set(messageOffset);
}
if((inputValue & 1) == 1)
{
message.set(messageOffset + 1);
}
}
}
//Transfer the corrected error count to the message
message.setCorrectedBitCount(path.getError());
return message;
}
/**
* Extracts (49) four-bit symbols (196 bits / 4) from the deinterleaved P25 binary message
*
* @param encodedMessage that deinterleaved and contains 196-bits of 1/2 TCM encoding
* @return symbols
*/
public int[] getSymbols(BinaryMessage encodedMessage)
{
//Ensure we have an integral number of transmitted symbols (nibbles) in the message
if(encodedMessage.size() % getOutputBitLength() != 0)
{
throw new IllegalArgumentException("Encoded message must contain an integral number of 4-bit symbols -" +
" message size: " + encodedMessage.size());
}
int[] symbols = new int[encodedMessage.size() / getOutputBitLength()];
int index = 0;
int pointer = 0;
while(index < encodedMessage.size())
{
int symbol = encodedMessage.getInt(index, index + (getOutputBitLength() - 1));
symbols[pointer++] = symbol;
index += getOutputBitLength();
}
return symbols;
}
/**
* Creates a P25 node
* @param inputValue for the specific trellis node
* @param transmittedOutputValue the actual transmitted output value
* @return created node
*/
@Override
protected Node createNode(int inputValue, int transmittedOutputValue)
{
return new P25_1_2_Node(inputValue, transmittedOutputValue);
}
/**
* Creates a P25 flushing node. P25 uses an input value of zero to flush the final output value out of the
* encoder.
*
* @param transmittedOutputValue the actual transmitted output value
* @return
*/
@Override
protected Node createFlushingNode(int transmittedOutputValue)
{
return new P25_1_2_Node(0, transmittedOutputValue);
}
/**
* Creates a P25 starting node. P25 uses a starting input value of zero for the initial node.
*/
@Override
protected Node createStartingNode()
{
return new P25_1_2_Node(0,0);
}
}

8
src/main/java/io/github/dsheirer/edac/trellis/ViterbiDecoder_3_4_DMR.java
View File

@@ -40,7 +40,7 @@ public class ViterbiDecoder_3_4_DMR extends ViterbiDecoder
* @param encodedMessage to decode that has already been deinterleaved.
* @return decoded message
*/
public BinaryMessage decode(BinaryMessage encodedMessage)
public CorrectedBinaryMessage decode(BinaryMessage encodedMessage)
{
int[] symbols = getSymbols(encodedMessage);
@@ -132,7 +132,7 @@ public class ViterbiDecoder_3_4_DMR extends ViterbiDecoder
@Override
protected Node createNode(int inputValue, int transmittedOutputValue)
{
return new DMRNode(inputValue, transmittedOutputValue);
return new DMR_3_4_Node(inputValue, transmittedOutputValue);
}
/**
@@ -145,7 +145,7 @@ public class ViterbiDecoder_3_4_DMR extends ViterbiDecoder
@Override
protected Node createFlushingNode(int transmittedOutputValue)
{
return new DMRNode(0, transmittedOutputValue);
return new DMR_3_4_Node(0, transmittedOutputValue);
}
/**
@@ -154,6 +154,6 @@ public class ViterbiDecoder_3_4_DMR extends ViterbiDecoder
@Override
protected Node createStartingNode()
{
return new DMRNode(0,0);
return new DMR_3_4_Node(0,0);
}
}

160
src/main/java/io/github/dsheirer/edac/trellis/ViterbiDecoder_3_4_P25.java Normal file
View File

@@ -0,0 +1,160 @@
/*******************************************************************************
* sdrtrunk
* Copyright (C) 2014-2017 Dennis Sheirer
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>
*
******************************************************************************/
package io.github.dsheirer.edac.trellis;
import io.github.dsheirer.bits.BinaryMessage;
import io.github.dsheirer.bits.CorrectedBinaryMessage;
import java.util.List;
public class ViterbiDecoder_3_4_P25 extends ViterbiDecoder
{
/**
* Viterbi decoder for P25 3/4 rate Trellis Coded Modulation (TCM) encoded messages.
*/
public ViterbiDecoder_3_4_P25()
{
super(3, 4);
}
/**
* Decodes a 3/4 rate trellis coded modulation (TCM) encoded P25 binary message containing 196 bits that have
* already been deinterleaved.
*
* @param encodedMessage to decode that has already been deinterleaved.
* @return decoded message
*/
public CorrectedBinaryMessage decode(BinaryMessage encodedMessage)
{
int[] symbols = getSymbols(encodedMessage);
Path mostLikelyPath = decode(symbols);
return getMessage(mostLikelyPath);
}
/**
* Extracts the decoded/corrected message from the surviving path argument.
*
* @param path to extract a message from
* @return corrected binary message.
*/
private static CorrectedBinaryMessage getMessage(Path path)
{
List<Node> mNodes = path.getNodes();
//Each node contains an input value excluding the starting(0) and final flushing(0) nodes.
CorrectedBinaryMessage message = new CorrectedBinaryMessage((mNodes.size() - 2) * 3);
if(mNodes.size() > 2)
{
for(int x = 1; x < mNodes.size() - 1; x++)
{
int inputValue = mNodes.get(x).getInputValue();
int messageOffset = 3 * (x - 1);
if((inputValue & 4) == 4)
{
message.set(messageOffset);
}
if((inputValue & 2) == 2)
{
message.set(messageOffset + 1);
}
if((inputValue & 1) == 1)
{
message.set(messageOffset + 2);
}
}
}
//Transfer the corrected error count to the message
message.setCorrectedBitCount(path.getError());
return message;
}
/**
* Extracts (49) four-bit symbols (196 bits / 4) from the deinterleaved P25 binary message
*
* @param encodedMessage that deinterleaved and contains 196-bits of 3/4 TCM encoding
* @return symbols
*/
public int[] getSymbols(BinaryMessage encodedMessage)
{
//Ensure we have an integral number of transmitted symbols (nibbles) in the message
if(encodedMessage.size() % getOutputBitLength() != 0)
{
throw new IllegalArgumentException("Encoded message must contain an integral number of 4-bit symbols -" +
" message size: " + encodedMessage.size());
}
int[] symbols = new int[encodedMessage.size() / getOutputBitLength()];
int index = 0;
int pointer = 0;
while(index < encodedMessage.size())
{
int symbol = encodedMessage.getInt(index, index + (getOutputBitLength() - 1));
symbols[pointer++] = symbol;
index += getOutputBitLength();
}
return symbols;
}
/**
* Creates a P25 node
*
* @param inputValue for the specific trellis node
* @param transmittedOutputValue the actual transmitted output value
* @return created node
*/
@Override
protected Node createNode(int inputValue, int transmittedOutputValue)
{
return new P25_3_4_Node(inputValue, transmittedOutputValue);
}
/**
* Creates a P25 flushing node. P25 uses an input value of zero to flush the final output value out of the
* encoder.
*
* @param transmittedOutputValue the actual transmitted output value
* @return
*/
@Override
protected Node createFlushingNode(int transmittedOutputValue)
{
return new P25_3_4_Node(0, transmittedOutputValue);
}
/**
* Creates a P25 starting node. P25 uses a starting input value of zero for the initial node.
*/
@Override
protected Node createStartingNode()
{
return new P25_3_4_Node(0, 0);
}
}